Escape sample coding in  palette-based video coding

ABSTRACT

In an example, a method of processing video data includes determining a value of a block-level syntax element that indicates, for all samples of a block of video data, whether at least one respective sample of the block is coded based on a color value of the at least one respective sample not being included in a palette of colors for coding the block of video data. The method also includes coding the block of video data based on the value.

This application claims the benefit of U.S. Provisional Application No.62/002,054, filed May 22, 2014, U.S. Provisional Application No.62/010,313, filed Jun. 10, 2014, U.S. Provisional Application No.62/015,240, filed Jun. 20, 2014, U.S. Provisional Application No.62/031,766 filed Jul. 31, 2014, U.S. Provisional Application No.62/040,978, filed Aug. 22, 2014, U.S. Provisional Application No.62/114,533, filed Feb. 10, 2015, and U.S. Provisional Application No.62/115,099, filed Feb. 11, 2015, the entire contents of each of whichare incorporated by reference herein.

TECHNICAL FIELD

This disclosure relates to video encoding and decoding.

BACKGROUND

Digital video capabilities can be incorporated into a wide range ofdevices, including digital televisions, digital direct broadcastsystems, wireless broadcast systems, personal digital assistants (PDAs),laptop or desktop computers, tablet computers, e-book readers, digitalcameras, digital recording devices, digital media players, video gamingdevices, video game consoles, cellular or satellite radio telephones,so-called “smart phones,” video teleconferencing devices, videostreaming devices, and the like. Digital video devices implement videocompression techniques, such as those described in the standards definedby MPEG-2. MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4. Part 10, AdvancedVideo Coding (AVC), the High Efficiency Video Coding (HEVC) standardpresently under development, and extensions of such standards. The videodevices may transmit, receive, encode, decode, and/or store digitalvideo information more efficiently by implementing such videocompression techniques.

Video compression techniques perform spatial (intra-picture) predictionand/or temporal (inter-picture) prediction to reduce or removeredundancy inherent in video sequences. For block-based video coding, avideo slice (i.e., a video frame or a portion of a video frame) may bepartitioned into video blocks. Video blocks in an intra-coded (I) sliceof a picture are encoded using spatial prediction with respect toreference samples in neighboring blocks in the same picture. Videoblocks in an inter-coded (P or B) slice of a picture may use spatialprediction with respect to reference samples in neighboring blocks inthe same picture or temporal prediction with respect to referencesamples in other reference pictures. Pictures may be referred to asframes, and reference pictures may be referred to as reference frames.

Spatial or temporal prediction results in a predictive block for a blockto be coded. Residual data represents pixel differences between theoriginal block to be coded and the predictive block. An inter-codedblock is encoded according to a motion vector that points to a block ofreference samples forming the predictive block, and the residual dataindicates the difference between the coded block and the predictiveblock. An intra-coded block is encoded according to an intra-coding modeand the residual data. For further compression, the residual data may betransformed from the pixel domain to a transform domain, resulting inresidual coefficients, which then may be quantized. The quantizedcoefficients, initially arranged in a two-dimensional array, may bescanned in order to produce a one-dimensional vector of coefficients,and entropy coding may be applied to achieve even more compression.

SUMMARY

Techniques of this disclosure relate to palette-based video coding. Forexample, in palette-based coding, a video coder (a video encoder orvideo decoder) may form a “palette” as a table of colors forrepresenting the video data of the particular area (e.g., a givenblock). Palette-based coding may be especially useful for coding areasof video data having a relatively small number of colors. Rather thancoding actual pixel values (or their residuals), the video coder maycode palette indices for one or more of the pixels that relate thepixels with entries in the palette representing the colors of thepixels. The techniques described in this disclosure may includetechniques for various combinations of one or more of signalingpalette-based coding modes, transmitting palettes, deriving palettes,and transmitting palette-based coding maps and other syntax elements.

In an example, a method of processing video data includes determining avalue of a block-level syntax element that indicates, for all samples ofa block of video data, whether at least one respective sample of theblock is coded based on a color value of the at least one respectivesample not being included in a palette of colors for coding the block ofvideo data, and coding the block of video data based on the value.

In another example, a device for processing video data includes a memoryconfigured to store a block of samples of video data, and one or moreprocessors configured to determine a value of a block-level syntaxelement that indicates, for all the samples of the block of video data,whether at least one respective sample of the block is coded based on acolor value of the at least one respective sample not being included ina palette of colors for coding the block of video data, and code theblock of video data based on the value.

In another example, an apparatus for processing video data includesmeans for determining a value of a block-level syntax element thatindicates, for all samples of a block of video data, whether at leastone respective sample of the block is coded based on a color value ofthe at least one respective sample not being included in a palette ofcolors for coding the block of video data, and means for coding theblock of video data based on the value.

In another example, a non-transitory computer-readable medium has storedthereon instructions that, when executed, cause one or more processorsto determine a value of a block-level syntax element that indicates, forall samples of a block of video data, whether at least one respectivesample of the block is coded based on a color value of the at least onerespective sample not being included in a palette of colors for codingthe block of video data, and code the block of video data based on thevalue.

In another example, a method of processing video data includes coding atleast one of data that indicates a maximum palette size of a palette ofcolor values for coding a block of video data or data that indicates amaximum palette predictor size of a palette predictor for determiningthe palette of color values, and coding the block of video data inaccordance with the data.

In another example, a device for processing video data includes a memoryconfigured to store a block of video data, and one or more processorsconfigured to code at least one of data that indicates a maximum palettesize of a palette of color values for coding the block of video data ordata that indicates a maximum palette predictor size of a palettepredictor for determining the palette of color values, and code theblock of video data in accordance with the data coded from thebitstream.

In another example, an apparatus for processing video data includesmeans for coding at least one of data that indicates a maximum palettesize of a palette of color values for coding a block of video data ordata that indicates a maximum palette predictor size of a palettepredictor for determining the palette of color values, and means forcoding the block of video data in accordance with the data.

In another example, a non-transitory computer-readable medium hasinstructions stored thereon that, when executed, cause one or moreprocessors to code at least one of data that indicates a maximum palettesize of a palette of color values for coding a block of video data ordata that indicates a maximum palette predictor size of a palettepredictor for determining the palette of color values, and code theblock of video data in accordance with the data.

In another example, a method of coding video data includes determining,for a pixel associated with a palette index that relates a value of thepixel to a color value in a palette of colors used for coding the pixel,a run length of a run of palette indices being coded with the paletteindex of the pixel, determining a maximum run length for a maximum runof palette indices able to be coded with the palette index of the pixel,and coding data that indicates the run length based on the determinedmaximum run length.

In another example, a device for coding video data includes a memoryconfigured to store a pixel of video data associated with a paletteindex that relates a value of the pixel to a color value in a palette ofcolors used for coding the pixel, and one or more processors configuredto determine, for the pixel, a run length of a run of palette indicesbeing coded with the palette index of the pixel, determining a maximumrun length for a maximum run of palette indices able to be coded withthe palette index of the pixel, and code data that indicates the runlength based on the determined maximum run length.

In another example, an apparatus for processing video data includesmeans for determining, for a pixel associated with a palette index thatrelates a value of the pixel to a color value in a palette of colorsused for coding the pixel, a run length of a run of palette indicesbeing coded with the palette index of the pixel, means for determining amaximum run length for a maximum run of palette indices able to be codedwith the palette index of the pixel, and means for coding data thatindicates the run length based on the determined maximum run length.

In another example, a non-transitory computer-readable medium hasinstructions stored thereon that, when executed, cause one or moreprocessors to determine, for a pixel associated with a palette indexthat relates a value of the pixel to a color value in a palette ofcolors used for coding the pixel, a run length of a run of paletteindices being coded with the palette index of the pixel, determine amaximum run length for a maximum run of palette indices able to be codedwith the palette index of the pixel, and code data that indicates therun length based on the determined maximum run length.

The details of one or more examples of the disclosure are set forth inthe accompanying drawings and the description below. Other features,objects, and advantages will be apparent from the description, drawings,and claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an example video coding systemthat may utilize the techniques described in this disclosure.

FIG. 2 is a block diagram illustrating an example video encoder that mayimplement the techniques described in this disclosure.

FIG. 3 is a block diagram illustrating an example video decoder that mayimplement the techniques described in this disclosure.

FIG. 4 is a conceptual diagram illustrating an example of determiningpalette entries for palette-based video coding, consistent withtechniques of this disclosure.

FIG. 5 is a conceptual diagram illustrating an example of determiningpalette indices to a palette for a block of pixels, consistent withtechniques of this disclosure.

FIG. 6 is a conceptual diagram illustrating an example of determining amaximum run length for a block of pixels, consistent with techniques ofthis disclosure.

FIG. 7 is a flowchart illustrating an example process for encoding ablock of video data based on one or more block-level syntax elementsthat indicate whether any sample of the block is encoded as escapesamples, consistent with techniques of this disclosure.

FIG. 8 is a flowchart illustrating an example process for decoding ablock of video data based on one or more block-level syntax elementsthat indicate whether any sample of the block is decoded as an escapesample, consistent with techniques of this disclosure.

FIG. 9 is a flowchart illustrating an example process for encoding ablock of video data based on one or more syntax elements that indicate amaximum palette size and a maximum palette predictor size, consistentwith techniques of this disclosure.

FIG. 10 is a flowchart illustrating an example process for encoding ablock of video data based on one or more syntax elements that indicate amaximum palette size and a maximum palette predictor size, consistentwith techniques of this disclosure.

FIG. 11 is a flowchart illustrating an example process for coding(encoding or decoding) data that indicates a run length of a run ofpixels based a maximum potential run length, consistent with techniquesof this disclosure.

DETAILED DESCRIPTION

Aspects of this disclosure are directed to techniques for video codingand compression. In particular, this disclosure describes techniques forpalette-based coding of video data. In traditional video coding, imagesare assumed to be continuous-tone and spatially smooth. Based on theseassumptions, various tools have been developed such as block-basedtransform, filtering, etc., and such tools have shown good performancefor natural content videos.

However, in applications like remote desktop, collaborative work andwireless display, computer generated screen content may be the dominantcontent to be compressed. This type of content tends to havediscrete-tone and feature sharp lines, and high contrast objectboundaries. The assumption of continuous-tone and smoothness may nolonger apply, and thus, traditional video coding techniques may beinefficient ways to compress the content.

This disclosure describes palette-based coding, which may beparticularly suitable for screen generated content coding (e.g., screencontent coding (SCC)). The techniques for palette-based coding of videodata may be used with one or more other coding techniques, such astechniques for inter- or intra-predictive coding. For example, asdescribed in greater detail below, an encoder or decoder, or combinedencoder-decoder (codec), may be configured to perform inter- andintra-predictive coding, as well as palette-based coding.

In some examples, the palette-based coding techniques may be configuredfor use with one or more video coding standards. For example, HighEfficiency Video Coding (HEVC) is a new video coding standard beingdeveloped by the Joint Collaboration Team on Video Coding (JCT-VC) ofITU-T Video Coding Experts Group (VCEG) and ISO/IEC Motion PictureExperts Group (MPEG). A recent HEVC text specification draft isdescribed in Bross et al., “High Efficiency Video Coding (HEVC) TextSpecification Draft 10 (for FDIS & Consent),” JCVC-L1003_v13, 12^(th)Meeting of JCT-VC of ITU-T SG16 WP 3 and ISO/IEC JCT 1/SC 29/WG 11,14-23 Jan. 2013 (“HEVC Draft 10”).

With respect to the HEVC framework, as an example, the palette-basedcoding techniques may be configured to be used as a coding unit (CU)mode. In other examples, the palette-based coding techniques may beconfigured to be used as a PU mode in the framework of HEVC.Accordingly, all of the following disclosed processes described in thecontext of a CU mode may, additionally or alternatively, apply to PU.However, these HEVC-based examples should not be considered arestriction or limitation of the palette-based coding techniquesdescribed herein, as such techniques may be applied to workindependently or as part of other existing or yet to be developedsystems/standards. In these cases, the unit for palette coding can besquare blocks, rectangular blocks or even regions of non-rectangularshape.

In palette-based coding, a particular area of video data may be assumedto have a relatively small number of colors. A video coder (a videoencoder or video decoder) may code a so-called “palette” as a table ofcolors for representing the video data of the particular area (e.g., agiven block). Each pixel may be associated with an entry in the palettethat represents the color of the pixel. For example, the video coder maycode an index that relates the pixel value to the appropriate value inthe palette.

In the example above, a video encoder may encode a block of video databy determining a palette for the block, locating an entry in the paletteto represent the value of each pixel, and encoding the palette withpalette indices (also referred to as palette index values) for thepixels relating the pixel value to the palette. A video decoder mayobtain, from an encoded bitstream, a palette for a block, as well aspalette indices for the pixels of the block. The video decoder mayrelate the palette indices of the pixels to entries of the palette toreconstruct the pixel values of the block. Pixels (and/or relatedpalette indices that indicate a pixel value) may generally be referredto as samples.

It is assumed that samples in the block are processed (e.g., scanned)using horizontal raster scanning order. For example, the video encodermay convert a two-dimensional block of palette indices into aone-dimensional array by scanning the palette indices using a horizontalraster scanning order. Likewise, the video decoder may reconstruct ablock of palette indices using the horizontal raster scanning order.Accordingly, this disclosure may refer to a previous sample as a samplethat precedes the sample currently being coded in the block in thescanning order. It should be appreciated that scans other than ahorizontal raster san, such as vertical raster scanning order, may alsobe applicable. The example above is intended provide a generaldescription of palette-based coding.

A palette typically includes entries numbered by an index andrepresenting color component (for example, RGB, YUV, or the like) valuesor intensities. Both a video encoder and a video decoder determine thenumber of palette entries, color component values for each palette entryand the exact ordering of the palette entries for the current block. Inthis disclosure, it is assumed that each palette entry specifies thevalues for all color components of a sample. However, the concepts ofthis disclosure are applicable to using a separate palette for eachcolor component.

In some examples, a palette may be composed using information frompreviously coded blocks. That is, a palette may contain predictedpalette entries predicted from the palette(s) used to code the previousblock(s). For example, as described in standard submission document WeiPu et al., “AHG10: Suggested Software for Palette Coding based onRExt6.0,” JCTVC-Q0094, Valencia, ES, 27 March-4 Apr. 2014 (hereinafterJCTVC-Q0094), a palette may include entries that are copied from apredictor palette. A predictor palette may include palette entries fromblocks previously coded using palette mode or other reconstructedsamples. For each entry in the predictor palette, a binary flag may becoded to indicate whether the entry associated with the flag is copiedto the current palette (e.g., indicated by flag=1). The string of binaryflags may be referred to as the binary palette prediction vector. Thepalette for coding a current block may also include a number of newpalette entries, which may be explicitly coded (e.g., separately fromthe palette prediction vector). An indication of the number of newentries may also be coded. A sum of the predicted entries and newentries may indicate the total palette size in for block.

As proposed JCTVC-Q0094, each sample in a block coded with apalette-based coding mode may be coded using one of the three palettemodes, as set forth below:

-   -   Escape mode: in this mode, the sample value is not included into        a palette as a palette entry and the quantized sample value is        signaled explicitly for all color components. It is similar to        the signaling of the new palette entries, although for new        palette entries, the color component values are not quantized.    -   CopyFromTop mode (also referred to as CopyAbove mode): in this        mode, the palette entry index for the current sample is copied        from the sample located directly above in a block.    -   Value mode (also referred to as Index mode): in this mode, the        value of the palette entry index is explicitly signaled.

As described herein, a palette entry index may be referred as a paletteindex or simply index. These terms can be used interchangeably todescribe techniques of this disclosure. In addition, as described ingreater detail below, a palette index may have one or more associatedcolor or intensity values. For example, a palette index may have asingle associated color or intensity value associated with a singlecolor or intensity component of a pixel (e.g., an Red component of RGBdata, a Y component of YUV data, or the like). In another example, apalette index may have multiple associated color or intensity values. Insome instances, palette-based coding may be applied to code monochromevideo. Accordingly, “color value” may generally refer to any color ornon-color component used to generate a pixel value.

For CopyFromTop and Value modes, a run value (which may also be referredto simply as run) may also be signaled. A run value may indicate anumber of consecutive samples (e.g., a run of samples) in a particularscan order in a palette-coded block that are coded together. In someinstances, the run of samples may also be referred to as a run ofpalette indices, because each sample of the run has an associated indexto a palette.

A run value may indicate a run of palette indices that are coded usingthe same palette-coding mode. For example, with respect to Value mode, avideo coder (a video encoder or video decoder) may code a palette index(also referred to as a palette index value or simply index value) and arun value that indicates a number of consecutive samples in a scan orderthat have the same palette index and that are being coded with thepalette index. With respect to CopyFromTop mode, the video coder maycode an indication that an index for the current sample value is copiedbased on an index of an above-neighboring sample (e.g., a sample that ispositioned above the sample currently being coded in a block) and a runvalue that indicates a number of consecutive samples in a scan orderthat also copy a palette index from an above-neighboring sample and thatare being coded with the palette index. Accordingly, in the examplesabove, a run of palette indices refers to a run of palette indiceshaving the same value or a run of palette indices that are copied fromabove-neighboring palette indices.

Hence, the run may specify, for a given mode, the number of subsequentsamples that belong to the same mode. In some instances, signaling anindex and a run value may be similar to run length coding. In an examplefor purposes of illustration, a string of consecutive palette indices ofa block may be 0, 2, 2, 2, 2, 5 (e.g., where each index corresponds to asample in the block). In this example, a video coder may code the secondsample (e.g., the first palette index of two) using Value mode. Aftercoding an index that is equal to 2, the video coder may code a run ofthree, which indicates that the three subsequent samples also have thesame palette index of two. In a similar manner, coding a run of fourpalette indices after coding an index using CopyFromTop mode mayindicate that a total of five palette indices are copied from thecorresponding palette indices in the row above the sample positioncurrently being coded.

The techniques described in this disclosure may include techniques forvarious combinations of one or more of signaling palette-based codingmodes, transmitting palettes, deriving palettes, and transmittingpalette-based coding maps and other syntax elements. In some examples,the techniques of this disclosure may be used to resolve potentialredundancies associated with the signaling of the palette modes, paletteindices, runs and palette sizes that are present in JCTVC-Q0094 (as wellas the reference software implementing the palette mode that wasuploaded with the contribution JCTVC-Q0094). Accordingly, as describedin greater detail below, the techniques of this disclosure may, in someinstances, improve efficiency and improve bitrate when coding video datausing a palette mode.

Certain aspects of this disclosure are directed to signalingpalette-based coding modes and, in particular, techniques associatedwith signaling escape samples. For example, escape samples (alsoreferred to as escape pixels) may be samples (or pixels) of a block thatdo not have a corresponding color represented in a palette for codingthe block. Accordingly, escape samples may not be reconstructed using acolor entry (or pixel value) from a palette. Instead, the color valuesfor escape samples are signaled in a bitstream separately from the colorvalues of the palette.

As described in greater detail below, a video coder (e.g., a videoencoder and a video decoder may code per-sample data that indicateswhether a sample of a palette-coded block is coded based on a color ofthe sample not being included in a palette for the block. e.g., usingthe process referred to as “Escape mode” above. In one example, thevideo coder may code a flag for each sample that indicates whether thesample is coded as an escape sample, e.g., using Escape mode (referredto herein as implicit escape signaling). In another example, the videocoder may code other syntax (such as an additional palette index, asdescribed below) for a sample that indicates that the sample is coded asan escape sample, e.g., using Escape mode (referred to herein asexplicit escape signaling).

According to aspects of this disclosure, for a palette-coded block, oneor more syntax elements may indicate, at block-level (e.g., a CU levelor LCU level), whether any sample of the block is coded based on a colorvalue of the sample not being included in the palette, e.g., coded as anescape sample. The one or more syntax elements may be referred to asblock-level escape syntax. For example, block-level syntax may refer tosyntax that is coded or determined while coding a block of video data,such as a CU or LCU. Block-level syntax may be included in a header orwith other data that is associated with the block (e.g., data that iscoded prior to or subsequent to a block that describes a characteristicof the block). In contrast, other syntax that is not block-level syntaxmay be included in a slice header or with individual pixels of videodata.

In one example, a video coder may be configured to code and/or determinea flag (which may be referred to as a block-level escape flag) thatindicates whether any sample of the block is coded based on a colorvalue not being included in the palette. For example, a flag value ofzero may indicate that none of the samples of the block are coded usingEscape mode. That is, the value of all samples of a block may bedetermined based on a color value that is included in a palette forcoding the block. A flag value of one may indicate that at least onesample of the block is coded using Escape mode. That is, the value of atleast one sample is not included in a palette for coding the block andmay be separately signaled. Hence, the flag may indicate, for allsamples of a block of video data, whether at least one sample of theblock has a color value that is not included in a palette for coding theblock.

As described in greater detail below, the block-level escape syntax mayresult, in some instances, in a bit savings. For example, by determiningwhether any samples of an entire block are coded as an escape sample,the video coder may be able to skip the coding of certain syntaxelements associated with escape samples. That is, in instances in whichthe syntax indicates no samples are coded as escape samples, the videocoder may not code any other syntax associated with escape samples forthe block (e.g., such as the per-sample syntax noted above). Asdescribed in greater detail below, the video coder may also skip thecoding of certain syntax when the syntax indicates that at least onesample of a block is coded as an escape sample based on a size of apalette for the block being coded. Accordingly, the techniques of thisdisclosure may improve bitrate and coding efficiency when coding videodata using palette-based coding.

Other aspects of this disclosure are directed to coding maximum paletteparameters for palette-mode. For example, a maximum palette size for apalette may typically be a static value that is defined at both a videoencoder and a video decoder. Likewise, a maximum size of a palettepredictor (used for predicting palettes, as described in greater detailbelow) may also be a static value that is defined at both a videoencoder and a video decoder. Hence, these maximum palette parameters maynot be changed, regardless of the particular characteristics of thevideo data being coded.

According to aspects of this disclosure, a video coder may be configuredto code data indicating a maximum palette size and/or a maximum palettepredictor size. For example, according to aspects of this disclosure,data that indicates a maximum palette size and/or a maximum palettepredictor size may be included in a parameter set, such as a sequenceparameter set (SPS). Accordingly, the video coder may code at least oneof data that indicates a maximum palette size of a palette of colorvalues for coding a block of video data or data that indicates a maximumpalette predictor size of a palette predictor for determining thepalette of color values.

Coding data that indicates a maximum palette size and/or a maximumpalette predictor size may provide flexibility, which may improve codingefficiency. For example, the techniques may allow a video coder to usepalettes and palette predictors of different sizes based on thecharacteristics of the video data being coded (e.g., based on abit-depth of the data, a block size, a profile or level associated withthe data, or the like). Accordingly, the maximum palette parameters maybe tailored to the video data being coded, such that relatively largermaximum palette parameters may be defined for blocks that may benefitfrom such parameters. In addition, relatively smaller maximum paletteparameters may be defined to reduce complexity associated withconstructing palettes for blocks less likely to benefit from therelatively larger parameters.

Other aspects of this disclosure are directed to techniques codingvarious syntax elements for palette-based video coding. For example, thetechniques of this disclosure include coding syntax for palette coding,such as a run value (also referred to as a run-length value) of paletteindices, a palette prediction vector, or other palette related syntax,using a code that considers a maximum potential value of the syntaxbeing coded. In some instances, according to aspects of this disclosure,the syntax may be coded using a form of Exponential Golomb code, asdescribed in greater detail below. The techniques may, in someinstances, reduce the number of bits needed to represent palette relatedsyntax.

FIG. 1 is a block diagram illustrating an example video coding system 10that may utilize the techniques of this disclosure. As used herein, theterm “video coder” refers generically to both video encoders and videodecoders. In this disclosure, the terms “video coding” or “coding” mayrefer generically to video encoding or video decoding. Video encoder 20and video decoder 30 of video coding system 10 represent examples ofdevices that may be configured to perform techniques for palette-basedvideo coding in accordance with various examples described in thisdisclosure. For example, video encoder 20 and video decoder 30 may beconfigured to selectively code various blocks of video data, such as CUsor PUs in HEVC coding, using either palette-based coding ornon-palette-based coding. Non-palette-based coding modes may refer tovarious inter-predictive temporal coding modes or intra-predictivespatial coding modes, such as the various coding modes specified by HEVCDraft 10.

As shown in FIG. 1, video coding system 10 includes a source device 12and a destination device 14. Source device 12 generates encoded videodata. Accordingly, source device 12 may be referred to as a videoencoding device or a video encoding apparatus. Destination device 14 maydecode the encoded video data generated by source device 12.Accordingly, destination device 14 may be referred to as a videodecoding device or a video decoding apparatus. Source device 12 anddestination device 14 may be examples of video coding devices or videocoding apparatuses.

Source device 12 and destination device 14 may comprise a wide range ofdevices, including desktop computers, mobile computing devices, notebook(e.g., laptop) computers, tablet computers, set-top boxes, telephonehandsets such as so-called “smart” phones, televisions, cameras, displaydevices, digital media players, video gaming consoles, in-car computers,or the like.

Destination device 14 may receive encoded video data from source device12 via a channel 16. Channel 16 may comprise one or more media ordevices capable of moving the encoded video data from source device 12to destination device 14. In one example, channel 16 may comprise one ormore communication media that enable source device 12 to transmitencoded video data directly to destination device 14 in real-time. Inthis example, source device 12 may modulate the encoded video dataaccording to a communication standard, such as a wireless communicationprotocol, and may transmit the modulated video data to destinationdevice 14. The one or more communication media may include wirelessand/or wired communication media, such as a radio frequency (RF)spectrum or one or more physical transmission lines. The one or morecommunication media may form part of a packet-based network, such as alocal area network, a wide-area network, or a global network (e.g., theInternet). The one or more communication media may include routers,switches, base stations, or other equipment that facilitatecommunication from source device 12 to destination device 14.

In another example, channel 16 may include a storage medium that storesencoded video data generated by source device 12. In this example,destination device 14 may access the storage medium, e.g., via diskaccess or card access. The storage medium may include a variety oflocally-accessed data storage media such as Blu-ray discs, DVDs,CD-ROMs, flash memory, or other suitable digital storage media forstoring encoded video data.

In a further example, channel 16 may include a file server or anotherintermediate storage device that stores encoded video data generated bysource device 12. In this example, destination device 14 may accessencoded video data stored at the file server or other intermediatestorage device via streaming or download. The file server may be a typeof server capable of storing encoded video data and transmitting theencoded video data to destination device 14. Example file serversinclude web servers (e.g., for a website), file transfer protocol (FTP)servers, network attached storage (NAS) devices, and local disk drives.

Destination device 14 may access the encoded video data through astandard data connection, such as an Internet connection. Example typesof data connections may include wireless channels (e.g., Wi-Ficonnections), wired connections (e.g., DSL, cable modem, etc.), orcombinations of both that are suitable for accessing encoded video datastored on a file server. The transmission of encoded video data from thefile server may be a streaming transmission, a download transmission, ora combination of both.

The techniques of this disclosure are not limited to wirelessapplications or settings. The techniques may be applied to video codingin support of a variety of multimedia applications, such as over-the-airtelevision broadcasts, cable television transmissions, satellitetelevision transmissions, streaming video transmissions, e.g., via theInternet, encoding of video data for storage on a data storage medium,decoding of video data stored on a data storage medium, or otherapplications. In some examples, video coding system 10 may be configuredto support one-way or two-way video transmission to support applicationssuch as video streaming, video playback, video broadcasting, and/orvideo telephony.

Video coding system 10 illustrated in FIG. 1 is merely an example andthe techniques of this disclosure may apply to video coding settings(e.g., video encoding or video decoding) that do not necessarily includeany data communication between the encoding and decoding devices. Inother examples, data is retrieved from a local memory, streamed over anetwork, or the like. A video encoding device may encode and store datato memory, and/or a video decoding device may retrieve and decode datafrom memory. In many examples, the encoding and decoding is performed bydevices that do not communicate with one another, but simply encode datato memory and/or retrieve and decode data from memory.

In the example of FIG. 1, source device 12 includes a video source 18, avideo encoder 20, and an output interface 22. In some examples, outputinterface 22 may include a modulator/demodulator (modem) and/or atransmitter. Video source 18 may include a video capture device, e.g., avideo camera, a video archive containing previously-captured video data,a video feed interface to receive video data from a video contentprovider, and/or a computer graphics system for generating video data,or a combination of such sources of video data.

Video encoder 20 may encode video data from video source 18. In someexamples, source device 12 directly transmits the encoded video data todestination device 14 via output interface 22. In other examples, theencoded video data may also be stored onto a storage medium or a fileserver for later access by destination device 14 for decoding and/orplayback.

In the example of FIG. 1, destination device 14 includes an inputinterface 28, a video decoder 30, and a display device 32. In someexamples, input interface 28 includes a receiver and/or a modem. Inputinterface 28 may receive encoded video data over channel 16. Displaydevice 32 may be integrated with or may be external to destinationdevice 14. In general, display device 32 displays decoded video data.Display device 32 may comprise a variety of display devices, such as aliquid crystal display (LCD), a plasma display, an organic lightemitting diode (OLED) display, or another type of display device.

Video encoder 20 and video decoder 30 each may be implemented as any ofa variety of suitable circuitry, such as one or more microprocessors,digital signal processors (DSPs), application-specific integratedcircuits (ASICs), field-programmable gate arrays (FPGAs), discretelogic, hardware, or any combinations thereof. If the techniques areimplemented partially in software, a device may store instructions forthe software in a suitable, non-transitory computer-readable storagemedium and may execute the instructions in hardware using one or moreprocessors to perform the techniques of this disclosure. Any of theforegoing (including hardware, software, a combination of hardware andsoftware, etc.) may be considered to be one or more processors. Each ofvideo encoder 20 and video decoder 30 may be included in one or moreencoders or decoders, either of which may be integrated as part of acombined encoder/decoder (CODEC) in a respective device.

This disclosure may generally refer to video encoder 20 “signaling” or“transmitting” certain information to another device, such as videodecoder 30. The term “signaling” or “transmitting” may generally referto the communication of syntax elements and/or other data used to decodethe compressed video data. Such communication may occur in real- ornear-real-time. Alternately, such communication may occur over a span oftime, such as might occur when storing syntax elements to acomputer-readable storage medium in an encoded bitstream at the time ofencoding, which then may be retrieved by a decoding device at any timeafter being stored to this medium.

In some examples, video encoder 20 and video decoder 30 operateaccording to a video compression standard, such as HEVC standardmentioned above, and described in HEVC Draft 10. In addition to the baseHEVC standard, there are ongoing efforts to produce scalable videocoding, multiview video coding, and 3D coding extensions for HEVC. Inaddition, palette-based coding modes, e.g., as described in thisdisclosure, may be provided for extension of the HEVC standard. In someexamples, the techniques described in this disclosure for palette-basedcoding may be applied to encoders and decoders configured to operationaccording to other video coding standards, such as the ITU-T-H.264/AVCstandard or future standards. Accordingly, application of apalette-based coding mode for coding of coding units (CUs) or predictionunits (PUs) in an HEVC codec is described for purposes of example.

In HEVC and other video coding standards, a video sequence typicallyincludes a series of pictures. Pictures may also be referred to as“frames.” A picture may include three sample arrays, denoted S_(L),S_(Cb) and S_(Cr). S_(L) is a two-dimensional array (i.e., a block) ofluma samples. So, is a two-dimensional array of Cb chrominance samples.S_(Cr) is a two-dimensional array of Cr chrominance samples. Chrominancesamples may also be referred to herein as “chroma” samples. In otherinstances, a picture may be monochrome and may only include an array ofluma samples.

To generate an encoded representation of a picture, video encoder 20 maygenerate a set of coding tree units (CTUs). Each of the CTUs may be acoding tree block of luma samples, two corresponding coding tree blocksof chroma samples, and syntax structures used to code the samples of thecoding tree blocks. A coding tree block may be an N×N block of samples.A CTU may also be referred to as a “tree block” or a “largest codingunit” (LCU). The CTUs of HEVC may be broadly analogous to themacroblocks of other standards, such as H.264/AVC. However, a CTU is notnecessarily limited to a particular size and may include one or morecoding units (CUs). A slice may include an integer number of CTUsordered consecutively in the raster scan.

To generate a coded CTU, video encoder 20 may recursively performquad-tree partitioning on the coding tree blocks of a CTU to divide thecoding tree blocks into coding blocks, hence the name “coding treeunits.” A coding block is an N×N block of samples. A CU may be a codingblock of luma samples and two corresponding coding blocks of chromasamples of a picture that has a luma sample array, a Cb sample array anda Cr sample array, and syntax structures used to code the samples of thecoding blocks. Video encoder 20 may partition a coding block of a CUinto one or more prediction blocks. A prediction block may be arectangular (i.e., square or non-square) block of samples on which thesame prediction is applied. A prediction unit (PU) of a CU may be aprediction block of luma samples, two corresponding prediction blocks ofchroma samples of a picture, and syntax structures used to predict theprediction block samples. Video encoder 20 may generate predictive luma,Cb and Cr blocks for luma, Cb and Cr prediction blocks of each PU of theCU.

Video encoder 20 may use intra prediction or inter prediction togenerate the predictive blocks for a PU. If video encoder 20 uses intraprediction to generate the predictive blocks of a PU, video encoder 20may generate the predictive blocks of the PU based on decoded samples ofthe picture associated with the PU.

If video encoder 20 uses inter prediction to generate the predictiveblocks of a PU, video encoder 20 may generate the predictive blocks ofthe PU based on decoded samples of one or more pictures other than thepicture associated with the PU. Video encoder 20 may use uni-predictionor bi-prediction to generate the predictive blocks of a PU. When videoencoder 20 uses uni-prediction to generate the predictive blocks for aPU, the PU may have a single motion vector (MV). When video encoder 20uses bi-prediction to generate the predictive blocks for a PU, the PUmay have two MVs.

After video encoder 20 generates predictive luma, Cb and Cr blocks forone or more PUs of a CU, video encoder 20 may generate a luma residualblock for the CU. Each sample in the CU's luma residual block indicatesa difference between a luma sample in one of the CU's predictive lumablocks and a corresponding sample in the CU's original luma codingblock. In addition, video encoder 20 may generate a Cb residual blockfor the CU. Each sample in the CU's Cb residual block may indicate adifference between a Cb sample in one of the CU's predictive Cb blocksand a corresponding sample in the CU's original Cb coding block. Videoencoder 20 may also generate a Cr residual block for the CU. Each samplein the CU's Cr residual block may indicate a difference between a Crsample in one of the CU's predictive Cr blocks and a correspondingsample in the CU's original Cr coding block.

Furthermore, video encoder 20 may use quad-tree partitioning todecompose the luma, Cb and Cr residual blocks of a CU into one or moreluma, Cb and Cr transform blocks. A transform block may be a rectangularblock of samples on which the same transform is applied. A transformunit (TU) of a CU may be a transform block of luma samples, twocorresponding transform blocks of chroma samples, and syntax structuresused to transform the transform block samples. Thus, each TU of a CU maybe associated with a luma transform block, a Cb transform block, and aCr transform block. The luma transform block associated with the TU maybe a sub-block of the CU's luma residual block. The Cb transform blockmay be a sub-block of the CU's Cb residual block. The Cr transform blockmay be a sub-block of the CU's Cr residual block.

Video encoder 20 may apply one or more transforms to a luma transformblock of a TU to generate a luma coefficient block for the TU. Acoefficient block may be a two-dimensional array of transformcoefficients. A transform coefficient may be a scalar quantity. Videoencoder 20 may apply one or more transforms to a Cb transform block of aTU to generate a Cb coefficient block for the TU. Video encoder 20 mayapply one or more transforms to a Cr transform block of a TU to generatea Cr coefficient block for the TU.

After generating a coefficient block (e.g., a luma coefficient block, aCb coefficient block or a Cr coefficient block), video encoder 20 mayquantize the coefficient block. Quantization generally refers to aprocess in which transform coefficients are quantized to possibly reducethe amount of data used to represent the transform coefficients,providing further compression. After video encoder 20 quantizes acoefficient block, video encoder 20 may entropy encoding syntax elementsindicating the quantized transform coefficients. For example, videoencoder 20 may perform Context-Adaptive Binary Arithmetic Coding (CABAC)on the syntax elements indicating the quantized transform coefficients.Video encoder 20 may output the entropy-encoded syntax elements in abitstream.

Video encoder 20 may output a bitstream that includes theentropy-encoded syntax elements. The bitstream may include a sequence ofbits that forms a representation of coded pictures and associated data.The bitstream may comprise a sequence of network abstraction layer (NAL)units. Each of the NAL units includes a NAL unit header and encapsulatesa raw byte sequence payload (RBSP). The NAL unit header may include asyntax element that indicates a NAL unit type code. The NAL unit typecode specified by the NAL unit header of a NAL unit indicates the typeof the NAL unit. A RBSP may be a syntax structure containing an integernumber of bytes that is encapsulated within a NAL unit. In someinstances, an RBSP includes zero bits.

Different types of NAL units may encapsulate different types of RBSPs.For example, a first type of NAL unit may encapsulate an RBSP for apicture parameter set (PPS), a second type of NAL unit may encapsulatean RBSP for a coded slice, a third type of NAL unit may encapsulate anRBSP for SEI, and so on. NAL units that encapsulate RBSPs for videocoding data (as opposed to RBSPs for parameter sets and SEI messages)may be referred to as video coding layer (VCL) NAL units.

Video decoder 30 may receive a bitstream generated by video encoder 20.In addition, video decoder 30 may parse the bitstream to decode syntaxelements from the bitstream. Video decoder 30 may reconstruct thepictures of the video data based at least in part on the syntax elementsdecoded from the bitstream. The process to reconstruct the video datamay be generally reciprocal to the process performed by video encoder20. For instance, video decoder 30 may use MVs of PUs to determinepredictive blocks for the PUs of a current CU. In addition, videodecoder 30 may inverse quantize transform coefficient blocks associatedwith TUs of the current CU. Video decoder 30 may perform inversetransforms on the transform coefficient blocks to reconstruct transformblocks associated with the TUs of the current CU. Video decoder 30 mayreconstruct the coding blocks of the current CU by adding the samples ofthe predictive blocks for PUs of the current CU to corresponding samplesof the transform blocks of the TUs of the current CU. By reconstructingthe coding blocks for each CU of a picture, video decoder 30 mayreconstruct the picture.

In some examples, video encoder 20 and video decoder 30 may beconfigured to perform palette-based coding. For example, inpalette-based coding, rather than performing the intra-predictive orinter-predictive coding techniques described above, video encoder 20 andvideo decoder 30 may code a so-called palette as a table of colors forrepresenting the video data of the particular area (e.g., a givenblock). Each pixel may be associated with an entry in the palette thatrepresents the color of the pixel. For example, video encoder 20 andvideo decoder 30 may code an index that relates the pixel value to theappropriate value in the palette.

In the example above, video encoder 20 may encode a block of video databy determining a palette for the block, locating an entry in the paletteto represent the value of each pixel, and encoding the palette withpalette indices for the pixels relating the pixel value to the palette.Video decoder 30 may obtain, from an encoded bitstream, a palette for ablock, as well as palette indices for the pixels of the block. Videodecoder 30 may relate the palette indices of the pixels to entries ofthe palette to reconstruct the pixel values of the block.

As noted above, video encoder 20 and video decoder 30 may use a numberof different palette coding modes to code palette indices of a palette.For example, video encoder 20 and video decoder 30 may use an Escapemode, a CopyFromTop mode (also referred to as CopyAbove mode), or aValue mode (also referred to as Index mode) to code palette indices of ablock. In general, coding a sample using “Escape mode” may generallyrefer coding a sample of a block that does not have a correspondingcolor represented in a palette for coding the block. As noted above,such samples may be referred to as escape samples or escape pixels.

Another example palette coding mode is described in a third screencontent coding core experiment, subtest B.6, as described in Yu-WenHuang et al., “Description of Screen Content Core Experiment 3 (SCCE3):Palette Mode,” JCTVC-Q1123, Valencia, ES, 27 March-4 Apr. 2014(hereinafter Q1123), another mode was introduced into the softwarereleased by Canon on 26^(th) May 2014. The macro for this mode was“CANON_NEW_RUN_LAST_TRANSITION” and may be referred to herein asTransition Run mode. The Transition Run may be similar to Value mode inthat video encoder 20 or video decoder 30 may code an index valuefollowed by a run specifying the number of subsequent samples that havethe same palette index.

The difference between Value mode and the Transition Run mode is thatthe palette index of the transition run mode is not signaled in thebitstream. Rather, video encoder 20 and video decoder 30 may infer thepalette index. As described herein, inferring a value may refer to thedetermination of a value without reference to dedicated syntax thatrepresents the value that is coded in a bitstream. That is, videoencoder 20 and video decoder 30 may infer a value without coding adedicated syntax element for the value in a bitstream. The inferredindex may be referred to as a transition index.

In some examples, there may be two ways of signaling the palette modes.A first technique for signaling palette modes may be referred to asexplicit escape signaling. For example, in JCTVC-Q0094, if the macro“PLT_REMOVE_ESCAPE_FLAG” is zero, video encoder 20 may explicitly encodean escape flag for each sample of a block to indicate whether a samplebeing coded in a block is coded in Escape mode. If the sample is notcoded with Escape mode, video encoder 20 may encode additional data toindicate whether the mode is CopyFromTop or Value. In some instances,the additional data may be a flag, referred to herein as an SPoint flag(e.g., an SPoint flag value of zero may indicate CopyFromTop mode and anSPoint flag value of one may indicate Value mode, or vice versa).

Hence, with the explicit escape signaling, the SPoint flag may be usedto indicate a particular run type for a run of pixel values associatedwith the indicated mode. For example, video encoder 20 may encode anSPoint flag to indicate whether the index currently being coded and therun of subsequent palette indices being coded in a run are coded usingCopyFromTop mode or Value mode. Video encoder 20 does not encode theescape flag (e.g., “PLT_REMOVE_ESCAPE_FLAG”) and the SPoint flag (whennecessary) for the subsequent run samples. That is, video encoder 20 andvideo decoder 30 may infer the values of the escape flag and SPoint flagfor samples included in a run. For example, video encoder 20 and videodecoder 30 may determine the value of the escape flag and SPoint flagfor samples included in the run without reference to dedicated syntaxthat represent such values in the bitstream.

A second technique for signaling palette modes may be referred to asimplicit escape signaling. For example, if the macro“PLT_REMOVE_ESCAPE_FLAG” from JCTVC-Q0094 is one, video encoder 20 andvideo decoder 30 may be configured to increase the number of paletteentries of a palette by one to accommodate a special index to thepalette that does not correspond to any palette entry. In some examples,video encoder 20 and video decoder 30 may include the additional indexas the last palette index in the increased palette for a given block.The additional index may be used as an indication of Escape mode.

When performing implicit escape signaling, video encoder 20 may encode,for a particular sample value of a block, data that represents theadditional index to indicate that the additional sample is coded as anescape sample (e.g., a sample that does not have a color valuerepresented in a palette for coding the block). Video encoder 20 mayalso encode the color value(s) of the escape sample. Accordingly, in thecase of implicit escape signaling, there are only two possible modes(e.g., CopyFromTop mode or Value mode (also referred to as Index mode))to be signaled using explicit syntax. For example, only the SPoint flagmay be signaled to distinguish between the modes. If a sample is codedin Value mode and the index for Value mode is equal to the escape index(e.g., the above-noted additional index to the palette), video encoder20 and video decoder 30 may infer the sample to be coded as an escapesample. In this case no run is signaled. When using the implicit escapesignaling with the Transition Run mode, the SPoint flag may take values0 (e.g., Value mode), 1 (e.g., CopyFromTop mode) or 2 (e.g., TransitionRun mode).

The techniques described in this disclosure may include techniques forvarious combinations of one or more of signaling palette-based codingmodes, transmitting palettes, deriving palettes, and transmittingpalette-based coding maps and other syntax elements. In some examples,the techniques of this disclosure may be used to resolve potentialredundancies associated with the signaling of the palette modes, paletteindices, runs and palette sizes that are present in JCTVC-Q0094 (as wellas the reference software implementing the palette mode that wasuploaded with the contribution JCTVC-Q0094).

In software associated with the techniques described in JCTVC-Q0094,certain signaling redundancies have already been considered and removed.For example, in JCTVC-Q0094, the SPoint flag is not signaled for samplesin the first row of the block, because a block coded with the palettemode cannot typically use reconstructed samples from anabove-neighboring block to predict the current block. Anabove-neighboring block may generally refer to a block that neighborsand is positioned above a block. Similarly, if the mode for a samplethat precedes a sample currently being coded is CopyFromTop, the modefor the current pixel cannot be CopyFromTop.

This disclosure, however, recognizes other signaling redundancies and/orinefficiencies, which can be removed altogether or selectively. Asdescribed in greater detail below, the techniques improve video codingbitrate efficiency without materially effecting distortion. As oneexample, if the sample directly above a current sample is an escapesample, video encoder 20 and video decoder 30 may be configured not tocode the current sample using CopyFromTop mode. In this case, videoencoder 20 may not signal the SPoint for the sample, and video encoder20 and video decoder 30 may infer the SPoint flag to be equal to Valuemode if needed.

In another example, according to the techniques of this disclosure, ifneither the previous sample nor the sample directly above the currentsample in a block are escape samples and the previous and above sampleshave the same palette index, video encoder 20 and video decoder 30 beconfigured not to code the current sample using CopyFromTop mode. Thisis because, for CopyFromTop mode, the index of the current sample wouldbe the same as the previous sample. If the mode for the previous samplewas Value mode, the run associated with Value mode would be extended byone to incorporate the current sample. On the other hand, if the modefor the previous sample was CopyFromTop, video encoder 20 and videodecoder 30 may be configured not to code the current sample usingCopyFromTop mode, as noted above. Thus, in this case, video encoder 20may not signal the SPoint flag for the current sample, and video encoder20 and video decoder 30 may infer the SPoint flag to be equal to Valuemode if needed.

In another example, according to the techniques of this disclosure, ifthe previous run is greater than or equal to a width of the block beingcoded minus one, video encoder 20 and video decoder 30 may be configurednot to code the current sample using CopyFromTop mode. Since CopyFromTopmode may not follow CopyFromTop mode, as described above, video encoder20 and video decoder 30 may infer that if the mode associated with theprevious sample is coded using CopyFromTop mode, the mode from thecurrent sample may not be coded using CopyFromTop mode. If the previousrun was coded using Value mode and the previous run was greater than orequal to the width of the block minus one, video encoder 20 and videodecoder 30 may be configured to determine that the palette indices forthe previous sample and the sample directly above the current sample arethe same (in a similar manner to the example described above). In thiscase, if the current sample may not have the same index, makingCopyFromTop mode impossible. Thus, in this case, video encoder 20 maynot signal the SPoint flag for the current sample, and video encoder 20and video decoder 30 may infer the SPoint flag to be equal to Value modeif needed.

In another example, according to aspects of this disclosure, if thepalette size is one for a block being coded when using explicit escapesignaling, video encoder 20 and video decoder 30 may be configured notto code certain palette indices, such as the palette indices describedin U.S. Provisional Application 61/845,824, filed Jul. 12, 2013, U.S.Provisional Application 61/899,048, filed Nov. 1, 2013, or U.S.Provisional Application 61/913,040, filed Dec. 6, 2013. In addition,video encoder 20 may be configured not to code the SPoint flag, as videoencoder 20 and video decoder 30 may infer the SPoint flag to be equal toValue mode if needed. This is because, if the current sample is notcoded as an escape sample (e.g., a sample that does not have a colorvalue represented in a palette for coding the block), the palette indexfor the current sample is already known and derived equal to zero (asthe only one possible palette index). In this case, only the run issignaled. It is not necessary to distinguish between CopyFromTop andValue modes, since both modes provide an identical result. Similarly,for the implicit escape signaling, when the palette size is two, videoencoder 20 may signal the palette indices to distinguish between Valueand Escape modes, but the signaling of the SPoint flag is not necessaryfor the same reasons as above.

The techniques of this disclosure may also be used to removeredundancies when using Value, CopyFromTop, and Transition Run Modes.Hence, the techniques may improve video coding bit rate efficiencywithout materially effecting distortion. In an example for purposes ofillustration, a current sample is coded in Value mode and the TransitionRun mode is not available for use (e.g., only CopyFromAbove and Valuemodes are available). In this example, when the mode for the previoussample is Value, the index of the current sample cannot be the same asthat of the previous sample, otherwise the current sample is includedinto the previous Value mode and the run for Value mode is incrementedby one. Similarly, when the mode for the previous sample is CopyFromTop,the index of the current sample to be coded cannot be the same as theone above, otherwise the current sample is coded with CopyFromTop modeand possibly the run for CopyFromTop mode would be incremented by one.

With the above-described relationship in mind, video encoder 20 andvideo decoder 30 may reduce the index for the current sample by one whenthe index is greater than the index for the previous sample (e.g., ifprevious sample is in Value mode) or the top sample (e.g., if previoussample is in CopyFromTop mode). This process is described in C. Gisquetet al., “AHG10: Palette Index Coding,” JCTVC-Q0064. Valencia, ES, 27March-4 Apr. 2014 (hereinafter JCTVC-Q0064). Also, the number of maximumpossible palette indices may be reduced by one, regardless whether theprevious condition is true (current index is greater than the previousleft or above palette indices). For example, when using a variablelength code (e.g., such as a truncated binary code) to code the index,the number of palette entries may be reduced by one.

According to aspects of this disclosure, alternatively or additionallyto the process described above, the index adjustment process in Valuemode may be further modified when using the Transition Run mode. Forexample, according to aspects of this disclosure, if the sample is notthe first sample in the block and the previous sample is not coded as anescape sample, video encoder 20 and video decoder 30 may perform theindex adjustment process described below.

According to aspects of this disclosure, if the previous sample is codedin Value or Transition Run mode, video encoder 20 and video decoder 30may be configured to decrease the number of palette entries by one. Inaddition, if the index value is greater than or equal to the index valueof the previous sample (index or transition index), video encoder 20 andvideo decoder 30 may be configured to decrease the current index valueby one. This disclosure may refer to this decremented value as theadjusted index value. Then, if the index value for the previous sampleis not equal to the transition index and the number of palette entriesis greater than one, video encoder 20 and video decoder 30 may beconfigured to set a variable “update” to one, otherwise set the “update”to zero. If update is equal to one, video encoder 20 and video decoder30 may further decrease the number of palette entries by one. Thisdisclosure may refer to the decremented number of palette entries as theadjusted palette size.

In addition, if the transition index is greater than or equal to theindex value for the previous sample, video encoder 20 and video decoder30 may be configured to decrease the transition index by one. Thisdisclosure may refer to the decremented transition index as the adjustedtransition index value. If update is equal to one and the adjusted indexvalue is greater than the adjusted transition index value, video encoder20 and video decoder 30 may be configured to further decrease theadjusted index value by one. Additionally, video encoder 20 and videodecoder 30 may be configured to perform the last index adjustment onlyif the adjusted palette size is greater than one. This is because theadjusted index value may only be signaled if the adjusted palette sizeis greater than one.

If the adjusted palette size is greater than one, video encoder 20 mayencode an indication of the adjusted index value, taking into accountthat the maximum possible number of palette indices may be equal to theadjusted palette size. In this case, video encoder 20 and video decoder30 may be configured to use truncated binarization for coding, such asthe truncated binary coding described herein.

In some examples, according to aspects of this disclosure, a similarprocess as the process described above may be performed by checking thepixel value and mode used for the pixel directly above the currentsample. That is, the process described above with respect to a pixelpositioned to the left of the current pixel may be performed instead forupper neighboring pixels, where the left sample value and mode describedabove is replaced with the above pixel value and mode.

For example, if the sample is not in the first row and the previoussample is coded in CopyFromTop mode and the sample above is not coded asan escape sample, video encoder 20 and video decoder 30 may beconfigured to decrease the number of palette entries by one. Inaddition, if the current index value is greater than or equal to theindex value of the sample directly above, video encoder 20 and videodecoder 30 may be configured to decrease the current index value by one.Again, this decremented index value may be referred to as the adjustedindex value. Then, if the index value for the sample directly above isnot equal to the transition index and the number of palette entries isgreater than one, video encoder 20 may set a variable update to one,otherwise set the update to zero.

If update is equal to one, video encoder 20 and video decoder 30 may beconfigured to further decrease the number of palette entries by one,which may be referred to as the adjusted palette size. In addition, ifthe transition index is greater than or equal to the index value for theabove sample, video encoder 20 and video decoder 30 may be configured todecrease the transition index by one, which may be referred to as theadjusted transition index value. If update is equal to zero and theadjusted index value is greater than the adjusted transition indexvalue, video encoder 20 and video decoder 30 may be configured todecrease the adjusted index value by one. Additionally, the last indexadjustment may be performed only if the adjusted palette size is greaterthan one, because the adjusted index value is typically only signaled ifthe adjusted palette size is greater than one.

If the adjusted palette size is greater than one, video encoder 20 maybe configured to encode an indication of the adjusted index value, andmay, in some examples, take the maximum possible number of paletteindices equal to the adjusted palette size into account. In this case,video encoder 20 and video decoder 30 may be configured to use truncatedbinarization, such as the truncated binary coding described herein.

The redundancy removal in the palette index signaling in connection withthe Transition Run mode is described above. However, these techniquesmay be combined with a Limited Run method, as described below and inU.S. Provisional Application No. 62/002,717 filed May 23, 2014 and U.S.Provisional Application No. 62/009,772, filed Jun. 9, 2014. In thiscase, above a certain palette index value, the run is always equal tozero and hence for those palette indices, video encoder 20 may beconfigured not to encode an indication of the run value. Rather, videoencoder 20 and video decoder 30 may be configured to derive the runvalue to be equal to zero. For this combination, the techniquesdescribed above with respect to Transition Run mode remain unchanged.That is, for example, the redundancy removal techniques described abovemay also be used with the Limited Run mode.

Additionally or alternatively, the techniques of this disclosure mayalso be combined with the Limited Run technique as proposed in standardsubmission document Guillaume Laroche et al., “AHG10: Run Coding forPalette Mode,” JCTVC-Q0066, Valencia, ES, 27 March-4 Apr. 2014(hereinafter JCTVC-Q0066). In this example, a limit index is alsospecified. However, one difference with the above-described limited runtechnique is that palette indices greater than a limit index may alsohave runs of one or greater. However, video encoder 20 may not signalthe runs. When implementing this second Limited Run technique, theredundancy removal techniques of this disclosure may only be applied ifthe index value of the previous pixel is less than or equal to the limitindex value, or the index value of the above pixel is less than or equalto the limit index value.

The above-described techniques are generally described with respect to avideo encoder (such as video encoder 20). On the decoder side (asimplemented, for example, by video decoder 30), using the sameconditions as on the encoder side, video decoder 30 may also adjust thenumber of palette entries and the transition index. Video decoder 30 maythen decode the index using the adjusted number of palette entries. Thedecoded index may be incremented (instead of decremented) using the sameconditions as on the encoder side.

Certain techniques described reduce redundancy in instances whichCopyFromTop mode is not possible, and hence signaling of the SPoint flagmay be modified such that video encoder 20 and video decoder 30 mayinfer Value mode. According to aspects of this disclosure, theredundancy reduction techniques may be extended to the case in whichTransition Run mode is also being used. In this case, CopyFromTop modeis not possible if any of the following conditions are true:

-   -   1. The sample is in the first row.    -   2. The mode for the previous sample is CopyFromTop.    -   3. The pixel above is coded in Escape mode and the sample is not        in the first row and the previous sample is not coded in        CopyFromTop mode.    -   4. The sample above and the previous sample have the same index        and the previous sample is not coded in Escape mode.

The techniques of this disclosure also provide an alternative forexplicitly signaling an escape flag for Escape palette mode. Forexample, instead of signaling the escape flag before the SPoint flagwith the explicit escape signaling, according to aspects of thisdisclosure, the order of the flags may be swapped while also changingthe semantics of those flags. In this case, video encoder 20 may signalthe SPoint flag first in a bitstream. In this example, an SPoint flagthat is equal to one may indicate Value mode, while an SPoint flag thatis equal to zero may indicate that the palette mode for a current sampleis either CopyFromTop or Escape. In addition, when the SPoint flag isequal to one, video encoder 20 may signal an escape flag todifferentiate between CopyFromTop mode and Escape mode.

In the examples above, video encoder 20 and video decoder 30 may beconfigured to use CABAC to code at least one of the above-describedflags or both of the above-described flags (e.g., the SPoint flag or theescape flag). Alternatively, video encoder 20 and video decoder 30 maybe configured to code such flags using CABAC bypass mode to reduce thenumber of context coded bins.

As described above, CopyFromTop mode may not be possible under certainconditions. In such cases, when using an alternate signaling method(e.g., such as swapping the flags), video encoder 20 may be configuredto only signal the SPoint flag without signaling the escape flag. Inthis case, the SPoint flag may have a different semantics. For example,an SPoint flag that is equal to one may still indicate that the mode isValue mode, but an SPoint flag that is equal to zero may indicate anEscape mode. If the SPoint flag is context-coded using CABAC, anadditional separate context may be used to code SPoint flag value incases when CopyFromTop mode is impossible. In the case of a palette sizebeing one and an escape mode being used, as described above, videoencoder 20 and video decoder 30 may be configured to skip the coding ofthe SPoint flag when using the alternate signaling method.

The techniques of this disclosure also provide another alternativesignaling technique (e.g., relative to JCTVC-Q0094) for signaling anescape flag for Escape palette mode. For example, in JCTVC-Q0094,certain signaling redundancies have been considered and removed in thereference software. As one example, when coding a current sample, if apalette mode for a previous sample is CopyFromTop, video encoder 20 andvideo decoder 30 may not code the current pixel using CopyFromTop mode.Similarly, if the mode for the previous sample is Value mode withpalette index “X”, video encoder 20 and video decoder 30 may not codethe current pixel using Value mode with the same palette index “X”. Atthe parsing stage (e.g., when parsing syntax elements from an encodedbitstream at video decoder 30), video decoder 30 checks the above-notedconditions to determine which syntax elements are allowed in order toread the bitstream properly. This checking process may become burdensomeif many such conditions are to be checked.

According to aspects of this disclosure, video encoder 20 and videodecoder 30 may be configured to “reuse” the above-noted redundancies toimplicitly signal Escape mode. For example, when coding a current sampleof a block, if a previous sample is coded using CopyFromTop mode and themode for the current pixel is also signaled as CopyFromTop, videodecoder 30 may infer that the mode for the current block is Escape mode.That is, video encoder 20 may use the redundancy of two samples in a rowbeing coded with CopyFromTop mode to signal Escape mode. Similarly, ifthe mode for the previous sample to a sample currently being coded isValue mode with palette index “X” and the signaled mode is Value modewith the same palette index “X”, video decoder 30 may infer the mode forthe current block to be Escape mode. Similarly, other redundanciesdescribed above may also be leveraged in this way.

In the examples described above, signaling Escape mode based onredundancies does not include all of the possible situations in whichvideo encoder 20 may signal Escape mode. Accordingly, these techniquesmay be used as a complementary way to signal Escape mode. In otherexamples, the techniques may be imposed on the bitstream, such thatEscape mode may only be signaled in these constrained situations.

The techniques of this disclosure also relate to signaling that a sampleis an escape sample for a palette size that is equal to zero. Forexample, according to aspects of this disclosure, if a palette size of apalette associated with a block currently being coded is equal to zerowhen using the explicit escape signaling, video encoder 20 and videodecoder 30 may be configured to infer that all of the samples in theblock are coded as escape samples. That is, video encoder 20 and videodecoder 30 may be configured to determine that all samples in the blockare coded as escape samples (e.g., samples that do not have a colorvalue represented in a palette for coding the block) without encoding ordecoding dedicated syntax that represents the Escape mode in abitstream. Likewise, if a palette size of a palette associated with ablock currently being coded is equal to one when using implicit escapesignaling (e.g., the only index of the palette is the additional indexused for signaling Escape mode, as described above), video encoder 20and video decoder 30 may be configured to infer that all of the samplesin the block are coded as escape samples.

In both of the above-described examples (e.g., for both explicit andimplicit escape signaling), video encoder 20 and video decoder 30 mayskip the coding of certain palette-based syntax for the rest of theblock. For example, for the explicit escape signaling, video encoder 20may not signal an escape flag for samples of the block. In addition, forboth the explicit and implicit escape signaling, video encoder 20 maynot signal the SPoint flag (for both implicit and explicit escapesignaling). That is, because all samples for the block may be inferredto be escape samples, video encoder 20 need not signal the Spoint flagto distinguish between CopyFromTop and Value modes. Video decoder 30 maylikewise skip decoding such syntax, which may improve bitrate and codingefficiency.

In an alternative example, video encoder 20 and video decoder 30 mayrestrict the palette size to be at least one in a normative fashion. Inthis example, video encoder 20 may be configured to modify the signalingof the palette size so that (palette size−1) is signaled. For example,when a palette predictor is used (e.g., as described in greater detailwith respect to the example of FIG. 4), for each entry of the palettepredictor, video encoder 20 may encode a one bit flag to indicatewhether respective palette predictor entries are included in a paletteof a current block. These entries are referred as the predicted paletteentries and are indicated by a palette prediction binary vector (e.g.,the string of one bit flags). Video encoder 20 may also signal thenumber of new palette entries following the predicted entries. In otherexamples, video encoder 20 may signal the number of new palette entriesprior to the predicted entries. In any case, if the number of thepredicted palette entries is zero, video encoder 20 and video decoder 30may be configured to code data that indicates (number of new paletteentries−1) instead of coding the number of new palette entries.

In another example, video encoder 20 and video decoder 30 may beconfigured to restrict the palette mode such that a palette size shallnot be equal to 0. For example, this restriction can be achieved as abitstream constraint, i.e. a bitstream cannot contain a palette codedblock with a palette size equal to zero.

The techniques of this disclosure also relate to signaling palette size.For example, the palette size for a current block (e.g., a CU currentlybeing coded by video encoder 20 or video decoder 30) may be explicitlysignaled (e.g., as disclosed, for example, in U.S. application Ser. No.14/244,688, filed Apr. 3, 2014 and U.S. application Ser. No. 14/244,711,filed Apr. 3, 2014). In such examples, the palette size includes bothpredicted palette entries (e.g., determined using a palette predictor)and new palette entries (e.g., as explicitly signaled in the bitstream).

According to aspects of this disclosure, if the palette size issignaled, there may be no need to signal the number of new entries, asvideo encoder 20 and video decoder 30 may be configured to derived thenumber of new palette entries for a block from the number of predictedentries and the palette size (e.g., palette size−number of predictedentries=number of new entries). In addition, video encoder 20 and videodecoder 30 may be configured to terminate the prediction of entries ofprevious palettes when the palette size is signaled and that signaledsize number is reached when constructing the palette for a currentblock.

In some examples, the palette size may be predicted from previouspalettes, and video encoder 20 may be configured to signal only thedifference. Video encoder 20 and video decoder 30 may be configured tocode the difference between the palette size and the predicted palettesize for a block using an exponential Golomb, truncated unary or fixedlength code. In some instances, video encoder 20 and video decoder 30may be configured to make the prediction depend on (e.g., based on) theblock size being coded. For example, for an 8×8 block, the palette sizemay be predicted from the palette associated with the latest 8×8 blockcoded using palette mode (e.g., the 8×8 most recently coded in scanningorder prior to the current block). Likewise, video encoder 20 and videodecoder 30 may be configured to predict a palette size for a 16×16 blockbased on a palette from a previously coded 16×16 block, and a similarrelationship may be extended to blocks of other sizes. Alternatively, inanother example, a palette size may be predicted from the latest blockcoded with less or equal size to the current block size.

The techniques of this disclosure also relate to signaling a maximumpalette size and/or a maximum palette predictor size. For example,according to aspects of this disclosure, video encoder 20 and videodecoder 30 may be configured to code data indicating a maximum palettesize and/or a maximum palette predictor size. In some examples, videoencoder 20 and video decoder 30 may be configured to code such data froman SPS. Coding data indicating a maximum palette size and/or a maximumpalette predictor size may provide flexibility, e.g., allowing videoencoder 20 and video decoder 30 to use palettes and palette predictorsof different sizes for different profiles, levels, bit-depths, blocksizes, or the like. In the context of a video coding standard, a profilemay correspond to a subset of algorithms, features, or tools andconstraints that apply to them. For example, a profile may be a subsetof an entire bitstream syntax that is specified by a particular. A levelmay correspond to the limitations of decoder resource consumption, suchas, for example, decoder memory and computation, which may be related tothe resolution of pictures, bitrate, and block processing rate. Aprofile may be signaled with a profile_idc (profile indicator) value,while a level may be signaled with a level_idc (level indicator) value.

According to aspects of this disclosure, video encoder 20 and videodecoder 30 may be configured to use information regarding a maximumpalette size to determine the elements and flags associated with thepalette mode, for example, in signaling the number of new paletteentries. As an example, a maximum possible palette size may be denotedby MAX_PLT_SIZE, which may be encoded by video encoder 20 and decoded byvideo decoder 30. Similarly, a maximum possible size of a palettepredictor vector may be denoted by MAX_PLT_PREDICTOR_SIZE, which may beencoded by video encoder 20 and decoder by video decoder 30.

As another example, according to aspects of this disclosure, videoencoder 20 and video decoder 30 may code data that indicates the numberof “ones” in a palette prediction binary vector (e.g., which mayrepresent the number of entries from the palette predictor being copiedto a palette for coding a current block). In some instances, videoencoder 20 and video decoder 30 may be configured to code a syntaxelement numPredPalette to indicate the number of the predicted paletteentries. If the value of numPredPalette is equal to the value ofMAX_PLT_SIZE (i.e., the maximum palette size), video encoder 20 andvideo decoder 30 may be configured to skip the coding of the number ofnew palette entries altogether. Otherwise, if the value ofnumPredPalette is less than the value of MAX_PLT_SIZE, video encoder 20and video decoder 30 may use a truncated binarization based on(MAX_PLT_SIZE−numPredPalette), which is the maximum possible value forthe number of new palette entries, to code data indicating the number ofnew entries.

In general, truncated binarization may include any technique that usesinformation about a maximum possible value of a particular parameterbeing signaled (e.g., such as the number of new palette entries) bydecreasing the length of some codewords used in the binarization methodof the parameter while maintaining unique decodability. For example,video encoder 20 and video decoder 30 may be configured to construct atruncated binary code using a maximum value of a given parameter (e.g.,such as the number of new palette entries). Example techniques fortruncated binary coding are described athttp://en.wikipedia.org/wiki/Truncated_binary_encoding.

Similarly, video encoder 20 and video decoder 30 may use truncated unaryor exponential Golomb or Golomb-Rice codes to signal and decode thenumber of new palette entries, based on a maximum possible value of thenumber of new palette entries. For example, if(MAX_PLT_SIZE−numPredPalette)=3, then video encoder 20 may use atruncated unary code to signal three as 000 instead of 0001 (e.g., aswould be signaled when using regular unary code). In case of thetruncated exponential Golomb or Golomb-Rice codes, video encoder 20 andvideo decoder 30 may be configured to reduce the length of the prefixfor the interval that contains the maximum value by one. Thus, videoencoder 20 and video decoder 30 may be configured to change the prefixfrom 000 . . . 001 to 000 . . . 000. Similarly, video encoder 20 andvideo decoder 30 may be configured to reduce the number of suffix bitsin the binarization method for that interval depending on the maximumvalue.

For large blocks (and/or large CUs), the palette size tends to be themaximum palette size. Therefore, in some cases, video encoder 20 andvideo decoder 30 may be configured to map the binarization of(MAX_PLT_SIZE−numPredPalette) in the inverse of the usual way, that is,with the shorter codeword lengths assigned to the larger values of(MAX_PLT_SIZE−numPredPalette), and the longer codeword lengths assignedto the smaller values of (MAX_PLT_SIZE−numPredPalette). In someexamples, instead of using O's followed by a 1 to signal unary/truncatedunary codes, or as a prefix of Golomb-Rice or exponential Golomb orconcatenated Golomb-Rice and exponential Golomb family of codes, videoencoder 20 and video decoder 30 may be configured to use l's followed a0.

Furthermore, other variations are possible. For example, video encoder20 and video decoder 30 may be configured to interpret the first bit insuch codes as a flag to indicate whether the number of new entries iszero or non-zero. Video encoder 20 and video decoder 30 may beconfigured to interpret the remaining bits as a number of new paletteentries minus 1. In an example for purposes of illustration, a maximumvalue of new palette entries may be eight and the number of new paletteentries may be three. Using truncated unary code, video encoder 20 andvideo decoder 30 may be configured to determine the binarization to be0001. If video encoder 20 and video decoder 30 are configured tointerpret the first bit as a flag (e.g., 0: one or more new paletteentries, 1: zero new palette entries), the rest of the bits (001)indicate that there are two new palette entries. When using truncatedcodes, video encoder 20 and video decoder 30 may be configured to adjustthe maximum value downwards by one.

In other examples, video encoder 20 and video decoder 30 may beconfigured to interpret the above-described flag in reverse. In thiscase, video encoder 20 and video decoder 30 may be configured tointerpret a flag value of I as one or more new palette entries and aflag value of 0 as zero new palette entries. In such a case, the bitsfor signaling three new palette entries with a maximum value of eightare 1001.

In other examples, the concept of the above-described flag may beextended to other codes such as exponential Golomb, Golomb-Rice, or thelike. For example, when the maximum value for new palette entries isnon-zero, video encoder 20 may be configured to signal a flag thatindicates whether there are non-zero new entries. If the flag indicatesthat there are non-zero new entries, number of new entries minus one maybe is signaled using exponential Golomb, Golomb-Rice, concatenation ofexponential Golomb and Golomb-Rice or similar codes or their truncatedversions. When truncated versions are used, the maximum value may beadjusted downwards by one.

In some examples, the flag may be context-coded using CABAC, whereas therest of the bins (e.g., for new palette entries minus 1) may be bypasscoded. Alternatively, the flag as well as the rest of the bins (for newpalette entries minus 1) may all be bypass coded. In some instances, afixed number of prefix bins from the code for the new palette entriesminus 1 may be context-coded using CABAC and the rest of the bins may bebypass coded.

According to aspects of this disclosure, as noted above, the syntaxelement MAX_PLT_SIZE may be signaled in a parameter set, such as an SPS.In other examples, the syntax element MAX_PLT_SIZE may be signaled in aVPS, picture parameter set (PPS), slice header, at a block level (e.g.,with syntax signaled for an LCU or CU), or elsewhere. In some examples,according to aspects of this disclosure, different maximum palette sizesmay be specified for different block sizes. In other examples, themaximum palette size may depend on a profile or the bit-depth of thevideo data being coded. For example, for a larger input bit-depth (orprofile bit-depth), the syntax element MAX_PLT_SIZE may be used tospecify a relatively larger maximum palette size. In still otherexamples, the maximum palette size may additionally or alternativelydepend on the chroma format of the video data being coded. For example,the syntax element MAX_PLT_SIZE may be used to specify a relativelysmaller maximum palette size for monochrome inputs than for 4:2:0 chromasub-sampling formats, which may, in turn, have smaller sizes than 4:4:4chroma sub-sampling formatted inputs.

According to aspects of this disclosure, instead of signaling the syntaxelement MAX_PLT_SIZE in the manner described above, data that indicates(MAX_PLT_SIZE−1) may be signaled, because a MAX_PLT_SIZE syntax elementthat is equal to zero may be invalid due to disabling the palettecompletely.

In an another example, instead of signaling a separate flag at the VPS.SPS, PPS, or slice header level to enable/disable palette mode, videoencoder 20 may be configured to only signal the MAX_PLT_SIZE syntaxelement. In this example, video encoder 20 and video decoder 30 may beconfigured to interpret a MAX_PLT_SIZE syntax element of 0 as disablingpalette mode. That is, upon receiving a palette size syntax element(e.g., the MAX_PLT_SIZE syntax element) video decoder 30 may determinethat palette mode has been disabled based on the syntax element. TheMAX_PLT_SIZE syntax element or (MAX_PLT_SIZE−1) may be signaled usingfixed length codes (assuming a normative limit on MAX_PLT_SIZE) orGolomb-Rice or exponential Golomb codes.

As noted above, the techniques of this disclosure also include codingdata that indicates a maximum palette predictor size. For example,according to aspects of this disclosure, video encoder 20 and videodecoder 30 may be configured to code a MAX_PLT_PREDICTOR_SIZE syntaxelement in the VPS, SPS, PPS, slice header, at a block level orelsewhere that indicates a maximum palette predictor size. In someexamples, instead of signaling the MAX_PLT_PREDICTOR_SIZE syntaxelement, (MAX_PLT_PREDICTOR_SIZE−1) may be signaled. In still otherexamples, video encoder 20 and video decoder 30 may be configured tocode other data that indicates a maximum palette predictor size.

In the particular examples described herein, the MAX_PLT_PREDICTOR_SIZEsyntax element or (MAX_PLT_PREDICTOR_SIZE−1) may be signaled using fixedlength codes (e.g., assuming a normative limit onMAX_PLT_PREDICTOR_SIZE) or Golomb Rice or exponential Golomb codes. Insome examples, video encoder 20 and video decoder 30 may be configuredto assume (e.g., automatically determine) that the size indicated by theMAX_PLT_PREDICTOR_SIZE syntax element is greater than or equal to amaximum palette size (e.g., as indicated by a MAX_PLT_SIZE syntaxelement). In this example, video encoder 20 and video decoder 30 may beconfigured to code (MAX_PLT_PREDICTOR_SIZE−MAX_PLT_SIZE) using fixedlength codes or Golomb-Rice or exponential Golomb codes. Accordingly,according to aspects of this disclosure, video encoder 20 and videodecoder 30 may be configured to code data indicating a delta (e.g.,difference) between the maximum palette predictor size and the maximumpalette size.

In examples in which the maximum palette size and maximum palettepredictor size are signaled at the SPS level, video encoder 20 and videodecoder 30 may be configured to code data that indicates the number ofnew entries using a truncated unary code. The number of new entries plusthe number of entries predicted from the palette predictor together maynot exceed the maximum palette size signaled in the SPS. However, if themaximum palette size signaled in the SPS is relatively large, the numberof new entries may exceed 31. In this instance, the truncated unary codeexceeds a 32-bit length, which may be undesirable for software andhardware implementations.

To address this, according to aspects of this disclosure, in oneexample, video encoder 20 and video decoder 30 may be configured torestrict the number of new entries so that length of the code forsignaling the number of new entries does not exceed 32. For example, ifa unary or truncated unary code is used to signal the number of newentries, the number of new entries may be restricted to 31. It should beunderstood that a length restriction of 32 is merely one example (e.g.,other length restrictions may alternatively be used).

In the HEVC screen content coding extensions text specification draft 2,(Rajan Joshi et al., “High Efficiency Video Coding (HEVC) Screen ContentCoding: Draft 2,” JCTVC-S1005, Sapporo, JP, 30 June-9 Jul. 2014(hereinafter JCTVC-S1005), truncated unary code is used to signal thenumber of new palette entries with the maximum value equal to maximumpalette size signaled in the SPS (palette_max_size) minus the number ofpalette entries that are predicted from the palette predictor. Using theproposed restriction, the maximum value may be modified to be thesmaller of 32 and the difference between the maximum palette sizesignaled in the SPS (palette_max_size) and the number of palette entriespredicted from the palette predictor. If such a modification to themaximum value is performed and the truncated unary coding of JCTVC-S1005is used, the maximum number of new palette entries may be 32 (instead of31) without the length of the code exceeding 32 bits.

In some examples, instead of truncated unary coding, if video encoder 20and video decoder 30 are configured use another code such as exponentialGolomb or its truncated version, the maximum allowable new paletteentries may be modified appropriately so that the length does not exceed32. If there are a number of values that have a codeword length of 32,video encoder 20 and video decoder 30 may be configured to choose thehighest of such value to be the maximum allowable value for the numberof new palette entries.

The restriction described herein on the maximum number new paletteentries may be made a normative restriction. For example, video encoder20 may be configured to generate a bitstream with the constraint andvideo decoder 30 may be configured to rely on the constraint in aconforming bitstream.

According to aspects of this disclosure, in one example, the semanticsof the palette_num_signaled_entries syntax element may be changedrelative to JCTVC-S1005 as follows: the syntax elementpalette_num_signaled_entries specifies the number of entries in thecurrent palette that are explicitly signaled. The value of the syntaxelement palette_num_signaled_entries shall be in the range of 0 to 31,inclusive. When the syntax element palette_num_signaled_entries is notpresent, it is inferred to be equal to 0.

In addition, the value of the variable CurrentPaletteSize specifies thesize of the current palette and is derived as follows:

If palette_share_flag [x0][y0] is equal to 1,

-   -   CurrentPaletteSize=PreviousPaletteSize (7-71)

Otherwise (palette_share_flag [x0][y0] is equal to 0)

-   -   CurrentPaletteSize=paletteNumPredictedEntries+

palette_num_signaled_entries(7-72)

In the example above, the value of CurrentPaletteSize shall be in therange of 0 to palette_max_size, inclusive.

According to aspects of this disclosure, if the maximum value ismodified in the above described manner, the value of the syntax elementpalette_num_signaled_entries may be modified such that the value shallbe in the range of 0 to 32, inclusive.

In another example, the maximum palette size may be signaled in the SPSand may be limited to 31. Limiting the size may be accomplished byenforcing an upper limit on the palette_max_size syntax element in thesemantics of the palette_max_size syntax element, such that thepalette_max_size syntax element specifies the maximum allowed palettesize. The value of the palette_max_size syntax element shall be in therange of 0 to 31, inclusive. When not present, the value of thepalette_max_size syntax element is inferred to be 0. In some examples,instead of 31, the value may be restricted to 32.

In another example, the maximum value of the palette_max_size syntaxelement may be restricted so that if the number of new palette entriesis equal to palette_max_size, video encoder 20 and video decoder 30 maybe configured to code the number of new palette entries using a codethat does not exceed 32 bits. In still another example, the maximumvalue of number of new palette entries may always be limited to 31,regardless of the code used to code the maximum value. In still anotherexample, the maximum value may be limited to 32.

The techniques of this disclosure also relate block-level escapesignaling (e.g., for a CU or LCU). For example, according to aspects ofthis disclosure, one or more syntax elements may indicate, atblock-level (e.g., a CU level), whether any of the samples of the blockare coded as an escape sample (e.g., a sample that does not have a colorvalue represented in a palette for coding the block). As noted above,the one or more syntax elements may be referred to as block-level escapesyntax. Again, block-level syntax may refer to syntax that is coded ordetermined with a block of video data, such as a CU or LCU, and notsyntax that may be included in a slice header or with individual pixelsof video data.

In instances in which at least one sample in a block of samples codedusing palette coding is coded as an escape sample, the techniques ofthis disclosure may be used to signal existence of such mode. In examplefor purposes of illustration, video encoder 20 and video decoder 30 maybe configured to code a flag (which may be referred to as a block-levelescape flag) that indicates whether any of the samples of the blockbeing coded are coded as an escape sample. In some instances, a flagvalue of zero may indicate that none of the samples or pixels of theblock are coded as escape samples. A flag value of one may indicate thatat least one sample or pixel of the block is coded as an escape sample.Hence, the block-level escape syntax may indicate, for all samples of ablock of video data, whether at least one sample of the block is codedwith without using an index to a palette of color values for the block,e.g., is coded using Escape mode.

According to aspects of this disclosure, the above described syntax mayachieve a bit savings relative to techniques of signaling escape sampleswithout a block-level indication. For example, in instances in which thesyntax indicates that no samples of a block are coded as escape samples,(e.g., the above-described flag is zero), video encoder 20 and videodecoder 30 may not code any other syntax associated with escape samplesfor the block. For example, with respect to the explicit escapesignaling described herein, video encoder 20 and video decoder 30 mayskip the coding of the sample-level escape mode flags. With respect tothe implicit escape signaling, video encoder 20 and video decoder 30 mayskip the coding of the additional index for the palette that indicatesthe escape sample. In this example, video encoder 20 and video decoder30 may only code an SPoint flag to distinguish between CopyFromTop modeand Value mode.

In some examples, the above-described flag may be signaled before thepalette entries for a block or CU currently being coded. In otherexamples, the above-described flag may be signaled after the paletteentries for a block or CU currently being coded. In some examples, videoencoder 20 and video decoder 30 may be configured to context code theabove-described flag. In such examples, video encoder 20 and videodecoder 30 may determine the contexts based on the block or CU sizeand/or the palette size for the current block or CU.

In some instances, the usage of escape samples may vary by block size.For example, the use of escape samples may be less prevalent inrelatively small blocks. In such instances, video encoder 20 and videodecoder 30 may be configured to determine that a block does not includeany escape samples and skip the coding of the above-described flag,thereby achieving a bit savings. For example, according to aspects ofthis disclosure, video encoder 20 and video decoder 30 may not code theblock-level flag for 8×8 blocks, where escape samples are much moreunlikely to be used than in larger block sizes. Similarly, for largeblock sizes (e.g., blocks of 64×64 pixels or larger), video encoder 20and video decoder 30 may be configured to determine that there arealways samples coded as escape samples. In such instances, video encoder20 and video decoder 30 may infer that the block-level escape flag for ablock is equal to one (e.g., at least one sample is an escape sample)and skip the coding of the block-level escape flag (e.g., an indicationof the block-level escape flag is not included in the bitstream).

The techniques of this disclosure also relate to coding a block ofsamples based on whether any samples of a block are coded as escapesamples. For example, as noted above, the techniques of this disclosuremay be used to indicate whether any samples are coded as escape samplesof a palette-coded block. In instances in which a block does not includeescape samples and when the size of a palette is one, video encoder 20and video decoder 30 may be configured to automatically determine thatall samples of the block have the same index value (e.g., the only entryof the palette). Video encoder 20 and video decoder 30 may also,therefore, skip the coding of other all other data used to determinepalette indices of the block. For example, video encoder 20 and videodecoder 30 may skip the coding of the SPoint flag, index signaling, anddata associated with runs of palette indices.

In an example for purpose of illustration, video decoder 30 may decode ablock-level escape flag that indicates that there are no samples in thecurrent block that are coded as escape samples (e.g., the flag is equalto zero). Video decoder 30 may also decode data that indicates thepalette for the block has a single entry (e.g., data indicating that thepalette size is one) or decode a palette that has a single entry. Inthis example, based on both conditions evaluating to true (e.g., nosamples are escape samples and the palette size is one), video decoder30 may automatically determine that all of the palette indices of theblock are equal to the single entry included in the palette. Videodecoder 30 may also skip the decoding of other data used to determinepalette indices of the block (e.g., such as SPoint flags, paletteindices, and run information).

In another example, according to aspects of this disclosure, when apalette size is one, runs of samples coded with palette index zero maybe terminated by escape samples. That is, a run of palette indices maybe interrupted by a position being coded as an escape sample. In thisexample, video encoder 20 and video decoder 30 may be configured to skipthe coding of the SPoint flag. In addition, in this example, videoencoder 20 and video decoder 30 may be configured to infer that the modefor the palette indices is Value mode as well as the index of Value mode(e.g., with only one entry in the palette, it may not be necessary tosignal the index for Value mode). In this example, video encoder 20 andvideo decoder 30 may infer that the sample immediately following a runis coded as an escape sample and skip the coding of escape relatedsyntax.

The techniques of this disclosure also relate to coding data indicatinga run value of a run of palette indices in palette coding. For example,as noted above, a run value may indicate a number of consecutive samples(e.g., a run of samples) in a particular scan order in a palette-codedblock that are coded together. In some instances, the run of samples mayalso be referred to as a run of palette indices, because each sample ofthe run has an associated index to a palette.

A run value may indicate a run of palette indices that are coded usingthe same palette-coding mode. For example, with respect to Value mode,video encoder 20 and video decoder 30 may code an index value and a runvalue that indicates a number of consecutive samples in a scan orderthat have the same index value and that are being coded with the indexvalue. With respect to CopyFromTop mode, video encoder 20 and videodecoder 30 may code an indication that an index for the current samplevalue is copied based on an index of an above-neighboring sample (e.g.,a sample that is positioned above the sample currently being coded in ablock) and a run value that indicates a number of consecutive samples ina scan order that also copy an index value from an above-neighboringsample and that are being coded with the index value.

For example, according to aspects of this disclosure, data indicating arun of palette indices in a block of video data may be coded based on amaximum possible run value for the block. That is, video encoder 20 andvideo decoder 30 may determine, for a pixel associated with a paletteindex that relates a value of the pixel to a color value in a palette, arun length of a run of palette indices being coded with the paletteindex of the pixel. Video encoder 20 and video decoder 30 may alsodetermine a maximum run length for a maximum run of palette indices ableto be coded with the palette index of the pixel. Video encoder 20 andvideo decoder 30 may then code data that indicates the run length basedon the determined maximum run length.

In an example for purposes of illustration, the total number of samplesin a block of video data may be N and each of samples may be indexedfrom 0 to (N−1). For the sample with position j, video encoder 20 andvideo decoder 30 may determine the maximum possible run value as(N−j−1). It should be noted that the run value indicates the number ofsubsequent samples being coded with the same palette coding mode (e.g.,Value mode or CopyFromTop mode) as the current sample. According toaspects of this disclosure, video encoder 20 and video decoder 30 may beconfigured to code data indicating the run value using a truncatedbinarization, taking into account the maximum possible run value. Ingeneral, truncated binarization may include any technique that usesinformation about a maximum possible value of a particular parameterbeing signaled (e.g., such as the number of new palette entries) bydecreasing the length of some codewords used in the binarization methodof the parameter while maintaining unique decodability. For example, atruncated binary code based on the maximum possible value of a run maybe used. Similarly, truncated unary or exponential Golomb or Golomb Ricecodes may be used to code and decode the run value, based on the maximumpossible value of a run. In some examples, the truncated binarizationmay be a combination of exponential Golomb and Golomb-Rice codes.

For example, a k^(th) order Exp-Golomb (EGk) code word is composed oftwo parts, a prefix and a suffix. For a given unsigned integer x, theprefix part of the EGk code word consists of a unary code correspondingto the value of:

${l(x)} = \left\lfloor {\log_{1}\left( {\frac{x}{2^{k}} + 1} \right)} \right\rfloor$

The suffix part is computed as the binary representation ofx−2^(k)(2^(l(x))−1) using k+1(x) bits.

As an example, Table 1 below includes several code words for EGO.

TABLE 1 EG0 Example Value x Code word (prefix-suffix) Code word length 01 1 1 01-0  3 2 01-1  3 3 001-00  5 4 001-01  5 5 001-10  5 6 001-11  5

In U.S. Provisional Application No. 62/019,223, filed Jun. 20, 2014, arun value is coded using 2^(nd) order Exp-Golomb code.

According to aspects of this disclosure, video encoder 20 and videodecoder 30 may be configured to code data indicating a run value using atruncated Exp-Golomb code. For example, a k^(th) order truncatedExp-Golomb (TEGk) code word is also composed of two parts, a prefix anda suffix. The prefix may be a unary prefix and the suffix may be abinary suffix. For example, for a given unsigned integer x and itslargest possible run value Xmax (e.g., the maximum run length), theprefix part of the EGk code word consists of a truncated unary codecorresponding to the value of:

${l(x)} = \left\lfloor {\log_{2}\left( {\frac{x}{2^{k}} + 1} \right)} \right\rfloor$

Specifically, the “trailing one” of the unary code can be avoided if:

$\left\lfloor {\log_{2}\left( {\frac{X\; \max}{2^{k}} + 1} \right)} \right\rfloor=={{l(x)}.}$

If the prefix is truncated, i.e.,

${\left\lfloor {\log_{2}\left( {\frac{X}{2^{k\;}} + 1} \right)} \right\rfloor=={l(x)}},$

the suffix part of TEGk is computed as the truncated binaryrepresentation of x−2k(2^(l(x))−1) using k+l(x) or k+l(x)−1 bits. Themaximum symbol value for the input of truncated binary code isXmax−2^(k)(2^(l(x))−1).

If the prefix is not truncated, the suffix part of TEGk is the same asEGk, i.e. binary representation of x−2k(2^(l(x))−1) using k+l(x) bits.As an example, Table 1 below includes several code words for TEG0.

TABLE 2 TEG0 Examples (X = 5) Value x Code word (prefix-suffix) Codeword length 0 1 1 1 01-0  3 2 01-1  3 3 00-0  3 4 00-01 4 5 00-10 4While the example of Table 2 above illustrates that prefix as being anumber of zeros followed by a trailing one (e.g., 00 . . . 1_), itshould be understood that in other examples, video encoder 20 and videodecoder 30 may code a number of ones followed by a trailing zero (e.g.,11 . . . 0).

According to aspects of this disclosure, video encoder 20 and videodecoder 30 may code a run value using the TEGk code described above. Insome examples, video encoder 20 and video decoder 30 may, for a currentpixel position in a block (or CU), determine the maximal run value Xmaxbased on the equation (Xmax=number of pixels in the current CU−currentposition in scanning order−1).

In another example, if the run value is first coded using a truncatedunary prefix, video encoder 20 and video decoder 30 may be configured toadjust the maximum run value accordingly, e.g., based on the truncatedvalue. For example, in some instances, video encoder 20 and videodecoder 30 may be configured to code a run value as a series of threeflags: greater than zero, greater than one, and greater than two. Inthis example, if the signaled run is greater than two, video encoder 20and video decoder 30 may code the remaining value (e.g., run value−3),potentially with another binarization method such as a combination ofexponential Golomb and Golomb-Rice codes or the TEGk code describedabove.

However, if (N−j−1) is equal to 0, video encoder 20 and video decoder 30do not code a run. Likewise, if (N−j−1) is equal to one, video encoder20 and video decoder 30 may only code the greater than zero flag.Likewise, if (N−j−1) is equal to two, video encoder 20 and video decoder30 may only code the greater than zero and the greater than one flags.Likewise, if (N−j−1) is equal to three, video encoder 20 and videodecoder 30 may only code the greater than zero flag, the greater thanone flag, and the greater than two flag. If (N−j−1) is more than three,in addition to greater than zero flag, the greater than one flag, andthe greater than two flag, video encoder 20 and video decoder 30 maycode the remaining value up to a maximum value of (N−j−4). In a similarway, the described process may be extended to use a number of flagsother than the three, for example, flags indicating a signaled valuegreater than number M, where M may be a non-negative value starting fromzero.

According to aspects of this disclosure, in the example above, videoencoder 20 and video decoder 30 may code the remaining run length usingthe TEGk code described above, with a maximum run value equal to (numberof pixels in the current CU−current position in scanning order−4). Inanother example, video encoder 20 and video decoder 30 may be configuredto code a flag that indicates whether the run value is greater than zeroand the remaining value as the run length minus one. For example, videoencoder 20 and video decoder 30 may code the greater than zero flag.Video encoder 20 and video decoder 30 may also code data that indicatesthe run length minus one using the TEGk code with a maximum value forthe TEGk code set equal to the maximum run length minus one. In oneexample, video encoder 20 and video decoder 30 may set k equal to zero,such that the TEGk code is a TEG0 code.

In other examples, video encoder 20 and video decoder 30 may use anyorder of the above-described TEG code for coding syntax elements forpalette coding. In an example, video encoder 20 and video decoder 30 mayset k equal to two, such that the TEGk code is a TEG2 code.

While the examples above are described with respect to coding a runvalue in palette coding, video encoder 20 and video decoder 30 may beconfigured to use the codes (such as the TEGk code) to code other syntaxfor palette coding. For example, as described in greater detail below,video encoder 20 and video decoder 30 may use the above-described codesfor coding a binary palette prediction vector, a CopyAbove run length,or other values.

In Joshi et al., “Non-SCCE3: Contexts for coding index runs,” JointCollaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 andISO/IEC JTC I/SC 29/WG 11, 18 Meeting, Sapporo, JP 30 June-9 Jul. 2014,JCTVC-R0174 (hereinafter JCTVC-R0174), the authors proposed to make thecontexts of the run-length codewords depend on the index if CopyLeftmode (e.g., which may operate in a similar manner to CopyFromTop mode)is used. However, in accordance with an example of this disclosure, ifthe current run mode is CopyFromAbove, video encoder 20 and videodecoder 30 may determine contexts for CABAC coding the run based on theindex value of the pixel that is positioned above the pixel currentlybeing coded. In this example, the above-neighboring pixel is outside ofthe current CU, video encoder 20 and video decoder 30 may determine thatthe corresponding index is equal to a predefined constant k. In someexamples, the constant k may be equal to zero.

In some examples, if the palette mode for coding a current pixel isCopyFromAbove mode, video encoder 20 and video decoder 30 may select oneof two candidate CABAC contexts to code the first bin of the run lengthcodeword based on whether the above-neighboring pixel has an index thatis equal to zero. As another example, if the palette mode for coding acurrent pixel is CopyPrevious mode, video encoder 20 and video decoder30 may select one of four candidate CABAC contexts to code the first binof the run length codeword based on based on whether the index is equalto zero, one, two, or larger than two.

FIG. 2 is a block diagram illustrating an example video encoder 20 thatmay implement the techniques of this disclosure. FIG. 2 is provided forpurposes of explanation and should not be considered limiting of thetechniques as broadly exemplified and described in this disclosure. Forpurposes of explanation, this disclosure describes video encoder 20 inthe context of HEVC coding. However, the techniques of this disclosuremay be applicable to other coding standards or methods.

Video encoder 20 represents an example of a device that may beconfigured to perform techniques for palette-based video coding inaccordance with various examples described in this disclosure. Forexample, video encoder 20 may be configured to selectively code variousblocks of video data, such as CUs or PUs in HEVC coding, using eitherpalette-based coding or non-palette-based coding. Non-palette-basedcoding modes may refer to various inter-predictive temporal coding modesor intra-predictive spatial coding modes, such as the various codingmodes specified by HEVC Draft 10. Video encoder 20, in one example, maybe configured to generate a palette having entries indicating pixelvalues, select pixel values in a palette to represent pixels values ofat least some pixel locations in a block of video data, and signalinformation associating at least some of the pixel locations in theblock of video data with entries in the palette corresponding,respectively, to the selected pixel values in the palette. The signaledinformation may be used by video decoder 30 to decode video data.

In the example of FIG. 2, video encoder 20 includes a predictionprocessing unit 100, video data memory 101, a residual generation unit102, a transform processing unit 104, a quantization unit 106, aninverse quantization unit 108, an inverse transform processing unit 110,a reconstruction unit 112, a filter unit 114, a decoded picture buffer116, and an entropy encoding unit 118. Prediction processing unit 100includes an inter-prediction processing unit 120 and an intra-predictionprocessing unit 126. Inter-prediction processing unit 120 includes amotion estimation unit and a motion compensation unit (not shown). Videoencoder 20 also includes a palette-based encoding unit 122 configured toperform various aspects of the palette-based coding techniques describedin this disclosure. In other examples, video encoder 20 may includemore, fewer, or different functional components.

Video data memory 101 may store video data to be encoded by thecomponents of video encoder 20. The video data stored in video datamemory 101 may be obtained, for example, from video source 18. Decodedpicture buffer 116 may be a reference picture memory that storesreference video data for use in encoding video data by video encoder 20,e.g., in intra- or inter-coding modes. Video data memory 101 and decodedpicture buffer 116 may be formed by any of a variety of memory devices,such as dynamic random access memory (DRAM), including synchronous DRAM(SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or othertypes of memory devices. Video data memory 101 and decoded picturebuffer 116 may be provided by the same memory device or separate memorydevices. In various examples, video data memory 101 may be on-chip withother components of video encoder 20, or off-chip relative to thosecomponents.

Video encoder 20 may receive video data. Video encoder 20 may encodeeach CTU in a slice of a picture of the video data. Each of the CTUs maybe associated with equally-sized luma coding tree blocks (CTBs) andcorresponding CTBs of the picture. As part of encoding a CTU, predictionprocessing unit 100 may perform quad-tree partitioning to divide theCTBs of the CTU into progressively-smaller blocks. The smaller block maybe coding blocks of CUs. For example, prediction processing unit 100 maypartition a CTB associated with a CTU into four equally-sizedsub-blocks, partition one or more of the sub-blocks into fourequally-sized sub-sub-blocks, and so on.

Video encoder 20 may encode CUs of a CTU to generate encodedrepresentations of the CUs (i.e., coded CUs). As part of encoding a CU,prediction processing unit 100 may partition the coding blocksassociated with the CU among one or more PUs of the CU. Thus, each PUmay be associated with a luma prediction block and corresponding chromaprediction blocks. Video encoder 20 and video decoder 30 may support PUshaving various sizes. As indicated above, the size of a CU may refer tothe size of the luma coding block of the CU and the size of a PU mayrefer to the size of a luma prediction block of the PU. Assuming thatthe size of a particular CU is 2N×2N, video encoder 20 and video decoder30 may support PU sizes of 2N×2N or N×N for intra prediction, andsymmetric PU sizes of 2N×2N, 2N×N, N×2N, N×N, or similar for interprediction. Video encoder 20 and video decoder 30 may also supportasymmetric partitioning for PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2Nfor inter prediction.

Inter-prediction processing unit 120 may generate predictive data for aPU by performing inter prediction on each PU of a CU. The predictivedata for the PU may include predictive blocks of the PU and motioninformation for the PU. Inter-prediction unit 121 may perform differentoperations for a PU of a CU depending on whether the PU is in an Islice, a P slice, or a B slice. In an I slice, all PUs are intrapredicted. Hence, if the PU is in an I slice, inter-prediction unit 121does not perform inter prediction on the PU. Thus, for blocks encoded inI-mode, the predicted block is formed using spatial prediction frompreviously-encoded neighboring blocks within the same frame.

If a PU is in a P slice, the motion estimation unit of inter-predictionprocessing unit 120 may search the reference pictures in a list ofreference pictures (e.g., “RefPicList0”) for a reference region for thePU. The reference region for the PU may be a region, within a referencepicture, that contains sample blocks that most closely corresponds tothe sample blocks of the PU. The motion estimation unit may generate areference index that indicates a position in RefPicList0 of thereference picture containing the reference region for the PU. Inaddition, the motion estimation unit may generate an MV that indicates aspatial displacement between a coding block of the PU and a referencelocation associated with the reference region. For instance, the MV maybe a two-dimensional vector that provides an offset from the coordinatesin the current decoded picture to coordinates in a reference picture.The motion estimation unit may output the reference index and the MV asthe motion information of the PU. The motion compensation unit ofinter-prediction processing unit 120 may generate the predictive blocksof the PU based on actual or interpolated samples at the referencelocation indicated by the motion vector of the PU.

If a PU is in a B slice, the motion estimation unit may performuni-prediction or bi-prediction for the PU. To perform uni-predictionfor the PU, the motion estimation unit may search the reference picturesof RefPicList0 or a second reference picture list (“RefPicList1”) for areference region for the PU. The motion estimation unit may output, asthe motion information of the PU, a reference index that indicates aposition in RefPicList0 or RefPicList1 of the reference picture thatcontains the reference region, an MV that indicates a spatialdisplacement between a prediction block of the PU and a referencelocation associated with the reference region, and one or moreprediction direction indicators that indicate whether the referencepicture is in RefPicList0 or RefPicList1. The motion compensation unitof inter-prediction processing unit 120 may generate the predictiveblocks of the PU based at least in part on actual or interpolatedsamples at the reference region indicated by the motion vector of thePU.

To perform bi-directional inter prediction for a PU, the motionestimation unit may search the reference pictures in RefPicList0 for areference region for the PU and may also search the reference picturesin RefPicList1 for another reference region for the PU. The motionestimation unit may generate reference picture indexes that indicatepositions in RefPicList0 and RefPicList1 of the reference pictures thatcontain the reference regions. In addition, the motion estimation unitmay generate MVs that indicate spatial displacements between thereference location associated with the reference regions and a sampleblock of the PU. The motion information of the PU may include thereference indexes and the MVs of the PU. The motion compensation unitmay generate the predictive blocks of the PU based at least in part onactual or interpolated samples at the reference regions indicated by themotion vectors of the PU.

In accordance with various examples of this disclosure, video encoder 20may be configured to perform palette-based coding. With respect to theHEVC framework, as an example, the palette-based coding techniques maybe configured to be used as a coding unit (CU) mode. In other examples,the palette-based coding techniques may be configured to be used as a PUmode in the framework of HEVC. Accordingly, all of the disclosedprocesses described herein (throughout this disclosure) in the contextof a CU mode may, additionally or alternatively, apply to PU. However,these HEVC-based examples should not be considered a restriction orlimitation of the palette-based coding techniques described herein, assuch techniques may be applied to work independently or as part of otherexisting or yet to be developed systems/standards. In these cases, theunit for palette coding can be square blocks, rectangular blocks or evenregions of non-rectangular shape.

Palette-based encoding unit 122, for example, may perform palette-baseddecoding when a palette-based encoding mode is selected, e.g., for a CUor PU. For example, palette-based encoding unit 122 may be configure togenerate a palette having entries indicating pixel values, select pixelvalues in a palette to represent pixels values of at least somepositions of a block of video data, and signal information associatingat least some of the positions of the block of video data with entriesin the palette corresponding, respectively, to the selected pixelvalues. Although various functions are described as being performed bypalette-based encoding unit 122, some or all of such functions may beperformed by other processing units, or a combination of differentprocessing units.

According to aspects of this disclosure, palette-based encoding unit 122may be configured to perform any combination of the techniques forpalette coding described herein. For example, according to aspects ofthis disclosure, palette-based encoding unit 122 may determine a valueof a syntax element that indicates, for all samples of a block of videodata, whether at least one respective sample of the block is coded witha first palette mode, where the first palette mode includes coding therespective sample of the block without using an index to a palette ofcolor values for the block. For example, palette-based encoding unit 122may determine a value of a block-level syntax element that indicateswhether any sample of the block is encoded as an escape sample (as asample that does not have a color value represented in a palette forcoding the block). In some examples, palette-based encoding unit 122 maydetermine an escape flag for a block that indicates whether any sampleof the block is encoded as an escape samples.

Additionally or alternatively, palette-based encoding unit 122 maydetermine at least one of data that indicates a maximum palette size ofa palette of color values for coding a block of video data or data thatindicates a maximum palette predictor size of a palette predictor fordetermining the palette of color values. For example, palette-basedencoding unit 122 may include such data in a parameter set, such as anSPS.

Additionally or alternatively, palette-based encoding unit 122 maydetermine, for a pixel associated with a palette index that relates avalue of the pixel to a color value in a palette of colors for codingthe pixel, a run length of a run of palette indices being coded with thepalette index of the pixel. That is palette-based encoding unit 122 maydetermine that an index value for a particular sample is being encodedwith a run of other subsequent palette indices. Palette-based encodingunit 122 may also determine a maximum run length for a maximum run ofpalette indices able to be encoded with the palette index of the pixel.As described in greater detail below with respect to entropy encodingunit 118, video encoder 20 may encode data that indicates the run lengthbased on the determined maximum run length.

Intra-prediction processing unit 126 may generate predictive data for aPU by performing intra prediction on the PU. The predictive data for thePU may include predictive blocks for the PU and various syntax elements.Intra-prediction processing unit 126 may perform intra prediction on PUsin I slices, P slices, and B slices.

To perform intra prediction on a PU, intra-prediction processing unit126 may use multiple intra prediction modes to generate multiple sets ofpredictive data for the PU. Intra-prediction processing unit 126 may usesamples from sample blocks of neighboring PUs to generate a predictiveblock for a PU. The neighboring PUs may be above, above and to theright, above and to the left, or to the left of the PU, assuming aleft-to-right, top-to-bottom encoding order for PUs, CUs, and CTUs.Intra-prediction processing unit 126 may use various numbers of intraprediction modes, e.g., 33 directional intra prediction modes. In someexamples, the number of intra prediction modes may depend on the size ofthe region associated with the PU.

Prediction processing unit 100 may select the predictive data for PUs ofa CU from among the predictive data generated by inter-predictionprocessing unit 120 for the PUs or the predictive data generated byintra-prediction processing unit 126 for the PUs. In some examples,prediction processing unit 100 selects the predictive data for the PUsof the CU based on rate/distortion metrics of the sets of predictivedata. The predictive blocks of the selected predictive data may bereferred to herein as the selected predictive blocks.

Residual generation unit 102 may generate, based on the luma, Cb and Crcoding block of a CU and the selected predictive luma, Cb and Cr blocksof the PUs of the CU, a luma. Cb and Cr residual blocks of the CU. Forinstance, residual generation unit 102 may generate the residual blocksof the CU such that each sample in the residual blocks has a value equalto a difference between a sample in a coding block of the CU and acorresponding sample in a corresponding selected predictive block of aPU of the CU.

Transform processing unit 104 may perform quad-tree partitioning topartition the residual blocks associated with a CU into transform blocksassociated with TUs of the CU. Thus, a TU may be associated with a lumatransform block and two chroma transform blocks. The sizes and positionsof the luma and chroma transform blocks of TUs of a CU may or may not bebased on the sizes and positions of prediction blocks of the PUs of theCU. A quad-tree structure known as a “residual quad-tree” (RQT) mayinclude nodes associated with each of the regions. The TUs of a CU maycorrespond to leaf nodes of the RQT.

Transform processing unit 104 may generate transform coefficient blocksfor each TU of a CU by applying one or more transforms to the transformblocks of the TU. Transform processing unit 104 may apply varioustransforms to a transform block associated with a TU. For example,transform processing unit 104 may apply a discrete cosine transform(DCT), a directional transform, or a conceptually similar transform to atransform block. In some examples, transform processing unit 104 doesnot apply transforms to a transform block. In such examples, thetransform block may be processed as a transform coefficient block.

Quantization unit 106 may quantize the transform coefficients in acoefficient block. The quantization process may reduce the bit-depthassociated with some or all of the transform coefficients. For example,an n-bit transform coefficient may be rounded down to an m-bit transformcoefficient during quantization, where n is greater than nm.Quantization unit 106 may quantize a coefficient block associated with aTU ofa CU based on a quantization parameter (QP) value associated withthe CU. Video encoder 20 may adjust the degree of quantization appliedto the coefficient blocks associated with a CU by adjusting the QP valueassociated with the CU. Quantization may introduce loss of information,thus quantized transform coefficients may have lower precision than theoriginal ones.

Inverse quantization unit 108 and inverse transform processing unit 110may apply inverse quantization and inverse transforms to a coefficientblock, respectively, to reconstruct a residual block from thecoefficient block. Reconstruction unit 112 may add the reconstructedresidual block to corresponding samples from one or more predictiveblocks generated by prediction processing unit 100 to produce areconstructed transform block associated with a TU. By reconstructingtransform blocks for each TU of a CU in this way, video encoder 20 mayreconstruct the coding blocks of the CU.

Filter unit 114 may perform one or more deblocking operations to reduceblocking artifacts in the coding blocks associated with a CU. Decodedpicture buffer 116 may store the reconstructed coding blocks afterfilter unit 114 performs the one or more deblocking operations on thereconstructed coding blocks. Inter-prediction processing unit 120 mayuse a reference picture that contains the reconstructed coding blocks toperform inter prediction on PUs of other pictures. In addition,intra-prediction processing unit 126 may use reconstructed coding blocksin decoded picture buffer 116 to perform intra prediction on other PUsin the same picture as the CU.

Entropy encoding unit 118 may receive data from other functionalcomponents of video encoder 20. For example, entropy encoding unit 118may receive coefficient blocks from quantization unit 106 and mayreceive syntax elements from prediction processing unit 100. Entropyencoding unit 118 may perform one or more entropy encoding operations onthe data to generate entropy-encoded data. For example, entropy encodingunit 118 may perform a context-adaptive variable length coding (CAVLC)operation, a CABAC operation, a variable-to-variable (V2V) length codingoperation, a syntax-based context-adaptive binary arithmetic coding(SBAC) operation, a Probability Interval Partitioning Entropy (PIPE)coding operation, an Exponential-Golomb encoding operation, or anothertype of entropy encoding operation on the data. Video encoder 20 mayoutput a bitstream that includes entropy-encoded data generated byentropy encoding unit 118. For instance, the bitstream may include datathat represents a RQT for a CU.

According to aspects of this disclosure, entropy encoding unit 118 maybe configured to code palette data using a TEGk code, as described abovewith respect to the example of FIG. 1. In particular, according toaspects of this disclosure, entropy encoding unit 118 may encode datathat indicates a run length for a run of palette indices based on adetermined maximum run length. In some examples, entropy encoding unit118 may encode the run length using a TEG2 code.

FIG. 3 is a block diagram illustrating an example video decoder 30 thatis configured to implement the techniques of this disclosure. FIG. 3 isprovided for purposes of explanation and is not limiting on thetechniques as broadly exemplified and described in this disclosure. Forpurposes of explanation, this disclosure describes video decoder 30 inthe context of HEVC coding. However, the techniques of this disclosuremay be applicable to other coding standards or methods.

Video encoder 20 represents an example of a device that may beconfigured to perform techniques for palette-based video coding inaccordance with various examples described in this disclosure. Forexample, video encoder 20 may be configured to selectively decodevarious blocks of video data, such as CUs or PUs in HEVC coding, usingeither palette-based coding or non-palette-based coding.Non-palette-based coding modes may refer to various inter-predictivetemporal coding modes or intra-predictive spatial coding modes, such asthe various coding modes specified by HEVC Draft 10. Video decoder 30,in one example, may be configured to generate a palette having entriesindicating pixel values, receive information associating at least somepixel locations in a block of video data with entries in the palette,select pixel values in the palette-based on the information, andreconstruct pixel values of the block based on the selected pixel valuesin the palette.

In the example of FIG. 3, video decoder 30 includes an entropy decodingunit 150, video data memory 151, a prediction processing unit 152, aninverse quantization unit 154, an inverse transform processing unit 156,a reconstruction unit 158, a filter unit 160, and a decoded picturebuffer 162. Prediction processing unit 152 includes a motioncompensation unit 164 and an intra-prediction processing unit 166. Videodecoder 30 also includes a palette-based decoding unit 165 configured toperform various aspects of the palette-based coding techniques describedin this disclosure. In other examples, video decoder 30 may includemore, fewer, or different functional components.

Video data memory 151 may store video data, such as an encoded videobitstream, to be decoded by the components of video decoder 30. Thevideo data stored in video data memory 151 may be obtained, for example,from channel 16, e.g., from a local video source, such as a camera, viawired or wireless network communication of video data, or by accessingphysical data storage media. Video data memory 151 may form a codedpicture buffer (CPB) that stores encoded video data from an encodedvideo bitstream. Decoded picture buffer 162 may be a reference picturememory that stores reference video data for use in decoding video databy video decoder 30, e.g., in intra- or inter-coding modes. Video datamemory 151 and decoded picture buffer 162 may be formed by any of avariety of memory devices, such as dynamic random access memory (DRAM),including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM),resistive RAM (RRAM), or other types of memory devices. Video datamemory 151 and decoded picture buffer 162 may be provided by the samememory device or separate memory devices. In various examples, videodata memory 151 may be on-chip with other components of video decoder30, or off-chip relative to those components.

A coded picture buffer (CPB) may receive and store encoded video data(e.g., NAL units) of a bitstream. Entropy decoding unit 150 may receiveencoded video data (e.g., NAL units) from the CPB and parse the NALunits to decode syntax elements. Entropy decoding unit 150 may entropydecode entropy-encoded syntax elements in the NAL units.

According to aspects of this disclosure, entropy decoding unit 150 maybe configured to decode palette data using a TEGk code, as describedabove with respect to the example of FIG. 1. In particular, according toaspects of this disclosure, entropy decoding unit 150 may decode datathat indicates a run length for a run of palette indices (e.g., a run ofindices having the same value or a run of indices that are copied fromabove-neighboring indices) based on a determined maximum run length. Insome examples, entropy decoding unit 150 may decode the run length usinga TEG2 code.

Prediction processing unit 152, inverse quantization unit 154, inversetransform processing unit 156, reconstruction unit 158, and filter unit160 may generate decoded video data based on the syntax elementsextracted from the bitstream. The NAL units of the bitstream may includecoded slice NAL units. As part of decoding the bitstream, entropydecoding unit 150 may extract and entropy decode syntax elements fromthe coded slice NAL units. Each of the coded slices may include a sliceheader and slice data. The slice header may contain syntax elementspertaining to a slice. The syntax elements in the slice header mayinclude a syntax element that identifies a PPS associated with a picturethat contains the slice.

In addition to decoding syntax elements from the bitstream, videodecoder 30 may perform a reconstruction operation on a non-partitionedCU. To perform the reconstruction operation on a non-partitioned CU,video decoder 30 may perform a reconstruction operation on each TU ofthe CU. By performing the reconstruction operation for each TU of theCU, video decoder 30 may reconstruct residual blocks of the CU.

As part of performing a reconstruction operation on a TU of a CU,inverse quantization unit 154 may inverse quantize, i.e., de-quantize,coefficient blocks associated with the TU. Inverse quantization unit 154may use a QP value associated with the CU of the TU to determine adegree of quantization and, likewise, a degree of inverse quantizationfor inverse quantization unit 154 to apply. That is, the compressionratio, i.e., the ratio of the number of bits used to represent originalsequence and the compressed one, may be controlled by adjusting thevalue of the QP used when quantizing transform coefficients. Thecompression ratio may also depend on the method of entropy codingemployed.

After inverse quantization unit 154 inverse quantizes a coefficientblock, inverse transform processing unit 156 may apply one or moreinverse transforms to the coefficient block in order to generate aresidual block associated with the TU. For example, inverse transformprocessing unit 156 may apply an inverse DCT, an inverse integertransform, an inverse Karhunen-Loeve transform (KLT), an inverserotational transform, an inverse directional transform, or anotherinverse transform to the coefficient block.

If a PU is encoded using intra prediction, intra-prediction processingunit 166 may perform intra prediction to generate predictive blocks forthe PU. Intra-prediction processing unit 166 may use an intra-predictionmode to generate the predictive luma, Cb and Cr blocks for the PU basedon the prediction blocks of spatially-neighboring PUs. Intra-predictionprocessing unit 166 may determine the intra prediction mode for the PUbased on one or more syntax elements decoded from the bitstream.

Prediction processing unit 152 may construct a first reference picturelist (RefPicList0) and a second reference picture list (RefPicList1)based on syntax elements extracted from the bitstream. Furthermore, if aPU is encoded using inter prediction, entropy decoding unit 150 mayextract motion information for the PU. Motion compensation unit 164 maydetermine, based on the motion information of the PU, one or morereference regions for the PU. Motion compensation unit 164 may generate,based on samples blocks at the one or more reference blocks for the PU,predictive luma, Cb and Cr blocks for the PU.

Reconstruction unit 158 may use the luma, Cb and Cr transform blocksassociated with TUs of a CU and the predictive luma, Cb and Cr blocks ofthe PUs of the CU, i.e., either intra-prediction data orinter-prediction data, as applicable, to reconstruct the luma, Cb and Crcoding blocks of the CU. For example, reconstruction unit 158 may addsamples of the luma, Cb and Cr transform blocks to corresponding samplesof the predictive luma, Cb and Cr blocks to reconstruct the luma, Cb andCr coding blocks of the CU.

Filter unit 160 may perform a deblocking operation to reduce blockingartifacts associated with the luma, Cb and Cr coding blocks of the CU.Video decoder 30 may store the luma, Cb and Cr coding blocks of the CUin decoded picture buffer 162. Decoded picture buffer 162 may providereference pictures for subsequent motion compensation, intra prediction,and presentation on a display device, such as display device 32 ofFIG. 1. For instance, video decoder 30 may perform, based on the luma,Cb, and Cr blocks in decoded picture buffer 162, intra prediction orinter prediction operations on PUs of other CUs.

In accordance with various examples of this disclosure, video decoder 30may be configured to perform palette-based coding. Palette-baseddecoding unit 165, for example, may perform palette-based decoding whena palette-based decoding mode is selected, e.g., for a CU or PU. Forexample, palette-based decoding unit 165 may be configured to generate apalette having entries indicating pixel values, receive informationassociating at least some pixel locations in a block of video data withentries in the palette, select pixel values in the palette-based on theinformation, and reconstruct pixel values of the block based on theselected pixel values in the palette. Although various functions aredescribed as being performed by palette-based decoding unit 165, some orall of such functions may be performed by other processing units, or acombination of different processing units.

Palette-based decoding unit 165 may receive palette coding modeinformation, and perform the above operations when the palette codingmode information indicates that the palette coding mode applies to theblock. When the palette coding mode information indicates that thepalette coding mode does not apply to the block, or when other modeinformation indicates the use of a different mode, video decoder 30 maydecode block of video data using a non-palette-based coding mode, e.g.,such an HEVC inter-predictive or intra-predictive coding mode. The blockof video data may be, for example, a CU or PU generated according to anHEVC coding process.

According to aspects of this disclosure, palette-based decoding unit 165may be configured to perform any combination of the techniques forpalette coding described herein. For example, according to aspects ofthis disclosure, palette-based decoding unit 165 may determine a valueof a syntax element that indicates, for all samples of a block of videodata, whether at least one respective sample of the block is coded witha first palette mode, where the first palette mode includes coding therespective sample of the block without using an index to a palette ofcolor values for the block. For example, palette-based decoding unit 165may determine a value of a block-level syntax element that indicateswhether any sample of the block is to be decoded as an escape sample(e.g., a sample that may not be reconstructed using a color entry fromthe palette). In some examples, palette-based decoding unit 165 maydetermine a one bit escape flag for a block that indicates whether anysamples of the block is to be decoded as an escape sample.

Additionally or alternatively, palette-based decoding unit 165 maydetermine at least one of data that indicates a maximum palette size ofa palette of color values for coding a block of video data or data thatindicates a maximum palette predictor size of a palette predictor fordetermining the palette of color values. For example, palette-baseddecoding unit 165 may decode such data from a parameter set, such as anSPS.

Additionally or alternatively, palette-based decoding unit 165 maydetermine, for a pixel associated with a palette index that relates avalue of the pixel to a color value in a palette of colors for codingthe pixel, a run length of a run of palette indices being coded togetherwith the palette index of the pixel (e.g., a run of indices having thesame value or a run of indices that are copied from above-neighboringindices). That is, palette-based decoding unit 165 may determine that anindex value for a particular sample is decoded with a run of othersubsequent palette indices. Palette-based decoding unit 165 may alsodetermine a maximum run length for a maximum run of palette indices ableto be decoded with the palette index of the pixel. As noted above withrespect to entropy decoding unit 150, video decoder 30 may decode datathat indicates the run length based on the determined maximum runlength.

FIG. 4 is a conceptual diagram illustrating an example of determining apalette for coding video data, consistent with techniques of thisdisclosure. The example of FIG. 4 includes a picture 178 having a firstcoding unit (CU) 180 that is associated with first palettes 184 and asecond CU 188 that is associated with second palettes 192. As describedin greater detail below and in accordance with the techniques of thisdisclosure, second palettes 192 are based on first palettes 184. Picture178 also includes block 196 coded with an intra-prediction coding modeand block 200 that is coded with an inter-prediction coding mode.

The techniques of FIG. 4 are described in the context of video encoder20 (FIG. 1 and FIG. 2) and video decoder 30 (FIG. 1 and FIG. 3) and withrespect to the HEVC video coding standard for purposes of explanation.However, it should be understood that the techniques of this disclosureare not limited in this way, and may be applied by other video codingprocessors and/or devices in other video coding processes and/orstandards.

In general, a palette refers to a number of pixel values that aredominant and/or representative for a CU currently being coded, CU 188 inthe example of FIG. 4. First palettes 184 and second palettes 192 areshown as including multiple palettes. In some examples, according toaspects of this disclosure, a video coder (such as video encoder 20 orvideo decoder 30) may code palettes separately for each color componentof a CU. For example, video encoder 20 may encode a palette for a luma(Y) component of a CU, another palette for a chroma (U) component of theCU, and yet another palette for the chroma (V) component of the CU. Inthis example, entries of the Y palette may represent Y values of pixelsof the CU, entries of the U palette may represent U values of pixels ofthe CU, and entries of the V palette may represent V values of pixels ofthe CU.

In other examples, video encoder 20 may encode a single palette for allcolor components of a CU. In this example, video encoder 20 may encode apalette having an i-th entry that is a triple value, including Yi, Ui,and Vi. In this case, the palette includes values for each of thecomponents of the pixels. Accordingly, the representation of firstpalettes 184 and 192 as a set of palettes having multiple individualpalettes is merely one example and not intended to be limiting.

In the example of FIG. 4, first palettes 184 includes three entries202-206 having entry index value 1, entry index value 2, and entry indexvalue 3, respectively. Entries 202-206 relate the palette indices topixel values including pixel value A, pixel value B, and pixel value C,respectively. As described herein, rather than coding the actual pixelvalues of first CU 180, a video coder (such as video encoder 20 or videodecoder 30) may use palette-based coding to code the pixels of the blockusing the palette indices 1-3. That is, for each pixel position of firstCU 180, video encoder 20 may encode an index value for the pixel, wherethe index value is associated with a pixel value in one or more of firstpalettes 184. Video decoder 30 may obtain the palette indices from abitstream and reconstruct the pixel values using the palette indices andone or more of first palettes 184. Thus, first palettes 184 aretransmitted by video encoder 20 in an encoded video data bitstream foruse by video decoder 30 in palette-based decoding.

According to aspects of this disclosure, a maximum palette size may besignaled for first palettes 184. For example, according to aspects ofthis disclosure, video encoder 20 and video decoder 30 may be configuredto code data indicating a maximum palette size, e.g., in terms of thenumber of entries that may be included in first palettes 184. In someexamples, one or more syntax elements that indicate the maximum palettesize (e.g., MAX_PLT_SIZE) may be included in an SPS that is active forCU 180. In other examples, one or more syntax elements that indicate themaximum palette size may be included in another parameter set, such as aVPS or PPS, or in header data such as slice header data or dataassociated with an LCU or CU.

In some examples, video encoder 20 and video decoder 30 may vary, usingthe one or more syntax elements that indicate the maximum palette size,the maximum palette size may be based on the particular profile, level,or bit-depth of the video data being coded. In other examples, videoencoder 20 and video decoder 30 may vary, using the one or more syntaxelements that indicate the maximum palette size, the maximum palettesize may be based on a size of the block being coded, such as CU 180.

In an example for purposes of illustration, video encoder 20 and videodecoder 30 may use the data indicating a maximum palette size whenconstructing first palettes 184 for CU 180. For example, video encoder20 and video decoder 30 may continue to add entries to first palettes184 until reaching the maximum palette size indicated by the data. Videoencoder 20 and video decoder 30 may then code CU 180 using theconstructed first palettes 184.

In some examples, video encoder 20 and video decoder 30 may determinesecond palettes 192 based on first palettes 184. For example, videoencoder 20 and/or video decoder 30 may locate one or more blocks fromwhich the predictive palettes, in this example, first palettes 184, aredetermined. The combination of entries being used for purposes ofprediction may be referred to as a predictor palette.

In the example of FIG. 4, second palettes 192 include three entries208-212 having entry index value 1, entry index value 2, and entry indexvalue 3, respectively. Entries 208-212 relate the palette indices topixel values including pixel value A, pixel value B, and pixel value D,respectively. In this example, video encoder 20 may code one or moresyntax elements indicating which entries of first palettes 184(representing a predictor palette, although the predictor palette mayinclude entries of a number of blocks) are included in second palettes192.

In the example of FIG. 4, the one or more syntax elements areillustrated as a vector 216. Vector 216 has a number of associated bins(or bits), with each bin indicating whether the palette predictorassociated with that bin is used to predict an entry of the currentpalette. For example, vector 216 indicates that the first two entries offirst palettes 184 (202 and 204) are included in second palettes 192 (avalue of “1” in vector 216), while the third entry of first palettes 184is not included in second palettes 192 (a value of “0” in vector 216).In the example of FIG. 4, the vector is a Boolean vector. The vector maybe referred to as a palette prediction vector.

In some examples, as noted above, video encoder 20 and video decoder 30may determine a palette predictor (which may also be referred to as apalette predictor table or palette predictor list) when performingpalette prediction. The palette predictor may include entries frompalettes of one or more neighboring blocks that are used to predict oneor more entries of a palette for coding a current block. Video encoder20 and video decoder 30 may construct the list in the same manner. Videoencoder 20 and video decoder 30 may code data (such as vector 216) toindicate which entries of the palette predictor are to be copied to apalette for coding a current block.

Thus, in some examples, previously decoded palette entries are stored ina list for use as a palette predictor. This list may be used to predictpalette entries in the current palette mode CU. A binary predictionvector may be signaled in the bitstream to indicate which entries in thelist are re-used in the current palette. In U.S. Provisional ApplicationNo. 62/018,461, filed Jun. 27, 2014, run length coding is used tocompress the binary palate predictor. In an example, the run-lengthvalue is coded using 0^(th) order Exp-Golomb code.

According to aspects of this disclosure, in some examples, video encoder20 and video decoder 30 (e.g., entropy encoding unit 118 and entropydecoding unit 150) may be configured to code (e.g., encode and decode,respectively) a binary palette prediction vector for a palette of ablock using a kth order truncated Exp-Golomb (TEGk) code, as describedabove with respect to the example of FIG. 1.

In some instances, video encoder 20 and video decoder 30 may beconfigured to code the binary palette prediction vector using the TEGkcode in conjunction with the techniques described in standard submissiondocument Seregin et al., “Non-SCCE3: Run-Length Coding for PalettePredictor,” JCTVC-R0228, Sapporo, JP, 30 June-9 Jul. 2014 (hereinafterJCTVC-R0228). In JCTVC-R0228, run-length coding is used to code the zeroelements in a binary vector with the following conditions and steps:

-   -   Run-length value equal to 1 indicates end of prediction    -   The end of prediction is not signaled for the last 1 in the        binary vector    -   The number of preceding zero elements is coded for every 1 in        the binary vector    -   If the number of zero elements is greater than 0, the number        plus one is signaled, due to the escape value of I    -   Run-length value is coded using 0-order Exponential Golomb code        In an example for purposes of illustration, a binary palette        prediction vector may be equal to {1100100010000}, indicating        that four entries (indicated by the four ones) of the palette        predictor are copied to the palette for coding a current block.        In this example, video encoder 20 and video decoder 30 may code        the vector as 0-0-3-4-1.

According to aspects of this disclosure, video encoder 20 and videodecoder 30 may code the binary palette prediction vector using a maximalrun value X for the vector, which may be equal to the number of paletteentries in the palette predictor list minus current position in scanningorder minus one). According to one example, video encoder 20 and videodecoder 30 use a TEG0 code for coding the run value.

The techniques of this disclosure also relate to signaling a maximumpalette predictor size. For example, according to aspects of thisdisclosure, video encoder 20 and video decoder 30 may be configured tocode data indicating a maximum palette predictor size, e.g., in terms ofthe number of bits that can be included in vector 216. In some examples,video encoder 20 and video decoder 30 may code data indicating themaximum palette predictor size relative to one or more other values. Forexample, according to aspects of this disclosure, video encoder 20 andvideo decoder 30 may be configured to code data indicating the maximumpalette predictor size as a delta (e.g., difference) between the maximumpalette predictor size and the maximum palette size. In some instances,video encoder and video decoder 30 may code the delta using at least oneof a fixed length code, a Golomb-Rice code, or an exponential Golombcode.

In some examples, syntax elements that indicate the maximum palettepredictor size may be included in an SPS. In other examples, syntaxelements that indicate the maximum palette predictor size may beincluded in another parameter set, such as an VPS or PPS, or in headerdata such as slice header data or data associated with an LCU or CU.

In some examples, video encoder 20 and video decoder 30 may vary, usingthe syntax elements that indicate the maximum palette predictor size,the maximum palette predictor size may be based on the particularprofile, level, or bit-depth of the video data being coded. In otherexamples, video encoder 20 and video decoder 30 may vary, using thesyntax elements that indicate the maximum palette predictor size, themaximum palette predictor size may be based on a size of the block beingcoded.

In an example for purposes of illustration, video encoder 20 and videodecoder 30 may use the data regarding the maximum palette predictor sizewhen constructing second palettes 192 for coding CU 188. For example,video encoder 20 and video decoder 30 may continue to add entries to apredictor palette (e.g., and bits to vector 216) until reaching amaximum palette predictor size, as indicated by the data. Video encoder20 and video decoder 30 may then use vector 216 to construct the secondpalettes 192 for CU 188.

FIG. 5 is a conceptual diagram illustrating an example of determiningpalette indices to a palette for a block of pixels, consistent withtechniques of this disclosure. For example, FIG. 5 includes a map 240 ofpalette indices that relate respective positions of pixels associatedwith the palette indices to an entry of palettes 244. For example, index1 is associated with Value A, index 2 is associated with Value B, andindex 3 is associated with Value C. In addition, when escape samples areindicated using implicit escape signaling, video encoder 20 and videodecoder 30 may also add an additional index to palettes 244, illustratedin FIG. 5 as index 4, which may indicate that samples of map 240associated with index 4 are escape samples. In this case, video encoder20 may encode (and video decoder 30 may obtain, from an encodedbitstream) an indication of an actual pixel value (or its quantizedversion) for a position in map 240 if the pixel value is not included inpalettes 244.

In some examples, video encoder 20 and video decoder 30 may beconfigured to code an additional map indicating which pixel positionsare associated with palette indices. For example, assume that the (i, j)entry in the map corresponds to the (i, j) position of a CU. Videoencoder 20 may encode one or more syntax elements for each entry of themap (i.e., each pixel position) indicating whether the entry has anassociated index value. For example, video encoder 20 may encode a flaghaving a value of one to indicate that the pixel value at the (i, j)location in the CU is one of the values in palettes 244.

Video encoder 20 may, in such an example, also encode a palette index(shown in the example of FIG. 5 as values 1-3) to indicate that pixelvalue in the palette and to allow video decoder to reconstruct the pixelvalue.

In instances in which palettes 244 include a single entry and associatedpixel value, video encoder 20 may skip the signaling of the index value.Video encoder 20 may encode the flag to have a value of zero to indicatethat the pixel value at the (i, j) location in the CU is not one of thevalues in palettes 244. In this example, video encoder 20 may alsoencode an indication of the pixel value for use by video decoder 30 inreconstructing the pixel value. In some instances, the pixel value maybe coded in a lossy manner.

The value of a pixel in one position of a CU may provide an indicationof values of one or more other pixels in other positions of the CU. Forexample, there may be a relatively high probability that neighboringpixel positions of a CU will have the same pixel value or may be mappedto the same index value (in the case of lossy coding, in which more thanone pixel value may be mapped to a single index value).

Accordingly, video encoder 20 may encode one or more syntax elementsindicating a number of consecutive pixels or palette indices in a givenscan order that are coded together. As noted above, the string ofpalette indices (or pixel values indicated by the palette indices) maybe referred to herein as a run. Video decoder 30 may obtain the syntaxelements indicating a run from an encoded bitstream and use the data todetermine the number of consecutive locations that have the same pixelor index value.

As noted above, runs may be used in conjunction with a CopyFromTop orValue mode. In an example for purposes of illustration, consider rows264 and 268 of map 240. Assuming a horizontal, left to right scandirection, row 264 includes three palette indices of “1,” two paletteindices of “2.” and three palette indices of “3.” Row 268 includes fivepalette indices of “1,” two palette indices of “3,” and one sample thatis not included in palettes 244 (represented by index 4, although asample-level escape flag may be used for explicit escape signaling),which may be referred to as an escape sample.

In this example, video encoder 20 may use CopyFromTop mode to encodedata for row 268. For example, video encoder 20 may encode one or moresyntax elements indicating that the first position of row 268 (the leftmost position of row 268) is the same as the first position of row 264.Video encoder 20 may also encode one or more syntax elements indicatingthat the next run of two consecutive entries in the scan direction inrow 268 are the same as the first position of row 264.

After encoding the one or more syntax elements indicating the firstposition of row 264 and the run of two entries (noted above), videoencoder 20 may encode the fourth and fifth positions in row 268 (fromleft to right), using Value mode. For example, video encoder 20 mayencode one or more syntax elements indicating a value of 1 for thefourth position and one or more syntax elements indicating a run of 1(e.g., Value mode). Hence, video encoder 20 encodes these two positionswithout reference to another line.

Video encoder 20 may then encode the first position having an indexvalue of 3 in row 268 using CopyFromTop mode relative to upper row 264.For example, video encoder 20 may signal a CopyFromTop mode and a runof 1. Accordingly, video encoder 20 may select between coding pixelvalues or palette indices of a line relative to other values of theline, e.g., using a run, coding pixel values or of a line relative tovalues of another line (or column), or a combination thereof. Videoencoder 20 may, in some examples, perform a rate/distortion optimizationto make the selection.

Video encoder 20 may then encode the escape sample for the final sampleof row 268 (from left to right), which is not included in first palettes244. For example, video encoder 20 may encode the final position of row268 as an escape sample. That is, video encoder 20 may encode anindication that the final position of row 268 is an escape sample (e.g.,index 4), as well as an indication of the sample value. Video decoder 30may obtain the above-described syntax from an encoded bitstream andreconstruct row 268 using such syntax.

As noted above, there may be two techniques to code the escape sample.For example, with explicit escape signaling, video encoder 20 and videodecoder 30 may code an explicit per-sample Escape mode flag for eachsample position of map 240. Ifa particular sample (such as the finalsample of row 268) is coded as an escape sample, video encoder 20 andvideo decoder 30 may code data that indicates the color value for theparticular sample. If the sample is not coded as an escape sample, videoencoder 20 and video decoder 30 may code additional data to indicatewhether the mode is CopyFromTop or Value, such as an SPoint flag.

With implicit escape signaling, video encoder 20 and video decoder 30may add an additional index to palettes 244 (entry index 4). Videoencoder 20 and video decoder 30 may use the additional index to palettes244 to indicate that a sample is coded as an escape sample, e.g., index4. The additional index, however, does not have an associated colorvalue. Rather, video encoder 20 and video decoder 30 also code colorvalues for each sample that is associated with the additional index. Ifthe sample is not coded as an escape sample, video encoder 20 and videodecoder 30 may code data to indicate whether the mode is CopyFromTop orValue, such as an SPoint flag.

According to aspects of this disclosure, video encoder 20 and videodecoder 30 may be configured to code one or more block-level syntaxelements that indicate, for all samples of a block of video data,whether at least one sample of the block is coded based on a color valuenot being included in a palette of colors for the block. With respect tothe example of FIG. 5, video encoder 20 and video decoder 30 may codeone or more syntax elements associated with map 240 that indicate thatat least one sample of map 240 is coded as an escape sample, i.e., thefinal sample of row 268.

In an example, the one or more syntax elements may be a block-levelescape flag (referred to below as simply “escape flag”). For example,video encoder 20 may encode an escape flag having a value of one toindicate that map 240 includes a sample coded as an escape sample.Likewise, video decoder 30 may decode an escape flag having a value ofone, which indicates that map 240 includes a sample coded as an escapesample. Accordingly, video encoder 20 may encode and video decoder 30may decode map 240 in accordance with the escape flag. For example,video encoder 20 and video decoder 30 may add index 4 to first palettes244, which may be used to represent samples coded as escape samples.Video encoder 20 and video decoder 30 may use this additional indexduring coding of map 240.

According to aspects of this disclosure, video encoder 20 may videodecoder 30 may be configured to skip the coding of certain syntax basedon the escape flag and the size of the palette being used to code aparticular block. That is, while the example of FIG. 5 illustrates firstpalettes 244 having three entries, in some instances, a palette forcoding a block of video data may include a single entry. In suchinstances, video encoder 20 and video decoder 30 may be configured toskip the coding of certain escape-related syntax based on the palettehaving a single entry and the escape flag indicating that no samples ofthe block are coded as escape samples.

For example, as in instances in which the escape flag indicates that nosamples of the block are escape samples and when the size of a paletteis one, video encoder 20 and video decoder 30 may be configured to inferthat all samples of the block have the same index value (e.g., the onlyentry of the palette). Video encoder 20 and video decoder 30 may also,therefore, skip the coding of other all other data used to determinepalette indices of the block.

In the example above, video encoder 20 and video decoder 30 mayexplicitly code block-level escape syntax (e.g., video encoder 20 mayencode an escape flag in the bitstream, and video decoder 30 may decodesuch a flag from the bitstream). However, in some examples, videoencoder 20 and video decoder 30 may infer (e.g., determine, withoutencoding or decoding the above-noted syntax element) the value of theblock-level escape syntax element based on the size of palettes 244 usedto code map 240. For example, video encoder 20 and video decoder 30 maymake a preliminary determination regarding palette size in order todetermine the value of the block-level escape flag. The block-levelescape syntax element may only be coded in the bitstream when thepalette size is greater than zero.

According to aspects of this disclosure, video encoder 20 and videodecoder 30 may initially determine a size of a palette for coding ablock of video data. Based on the size of the palette being zero, videoencoder 20 and video decoder 30 may determine that the escape flag isequal to one and that all samples of the block are coded as escapesamples, because there are no other palette entries available for codingsamples.

For example, as in instances in which the palette size is zero, videoencoder 20 and video decoder 30 may be configured to automaticallydetermine that all samples of the block have the same index value (e.g.,the additional entry of the palette associated with indicating escapesamples). Video encoder 20 and video decoder 30 may also, therefore,skip the coding of the escape flag as well as all other data used todetermine palette indices of the block.

FIG. 6 is a conceptual diagram illustrating an example of determiningmaximum run length for CopyFromAbove mode, assuming raster scanningorder, consistent with techniques of this disclosure. As noted above,the techniques of this disclosure include coding syntax for palettecoding using a code that accounts for a maximum potential value of thesyntax being coded.

In an example for purposes of illustration, video encoder 20 and videodecoder 30 may code a run-length of a run of palette indices (e.g., arun of indices having the same value or a run of indices that are copiedfrom above-neighboring indices). For example, video encoder 20 and videodecoder 30 may determine, for a current palette-coded sample, a runlength of a run of palette indices being coded together with the currentsample. Video encoder 20 and video decoder 30 may also determine amaximum run length for a maximum run of palette indices able to be codedwith the palette index of the pixel. Video encoder 20 and video decoder30 may then code data that indicates the run length based on thedetermined maximum run length

In some instances, according to aspects of this disclosure, the syntaxmay be coded using a form of Exponential Golomb code, such as the TEGkcode described herein. To use the TEGk code, video encoder 20 and videodecoder 30 may determine a maximum run-length as the number of pixels inthe current CU minus the current position in scanning order minus 1.

In some palette coding techniques, runs of pixels associated withCopyFromAbove mode (where a video coder copies an index of a pixel abovethe current pixel) is not permitted to include any escape pixels. Thatis, a video coder must stop a CopyFromAbove run if the current pixel'sabove-neighboring pixel is a pixel coded as an escape sample. Hence, themaximum CopyFromAbove run length is bounded by the distance between thecurrent pixel position and the position having an above-neighboringpixel that is escaped in the scanning order.

In an example for purposes of illustration, the starting position of aCopyFromAbove run in scanning order is A, the above-neighboring pixel tothe pixel in position A+L (L>0) (or in some examples, A+L (L>1)) iscoded as an escape sample, and the above-neighboring pixel to the pixelat position A+l (l<L) is not coded as an escape sample. If such a pixelL does not exist, video encoder 20 may assign L to the position afterthe last pixel in the block in scanning order. In accordance with thetechniques of this disclosure, video encoder 20 and video decoder 30 mayuse TEGk to code the run length for the CopyFromAbove mode with therestriction that the maximum coded run-length is no longer than L−1.Alternatively, if unary prefixes of greater than 0, greater than 1, andgreater than 2 are used when coding a run-length, video encoder 20 orvideo decoder 30 may set the maximum run-length of the run of the indexmap to be coded using TEGk to L−4.

In instances in which video decoder 30 or video encoder 20 cannotdetermine whether a pixel in a position corresponds to an escape pixelor not, video encoder 20 and video decoder 30 may process the pixel asif it is not coded as an escape sample, i.e. the maximum coded runlength does not stop. In the example of FIG. 6, if none of the pixelsencompassed by dashed lines 280 is coded as an escape sample, themaximum possible run length is 35 (i.e. the number of unshaded pixelpositions). If one or more of the pixels within dashed lines 280 iscoded as an escape sample, assuming that the pixel marked as the escapepixel (the pixel position with the “X”) is the first escape pixel withindashed lines 280 in scanning order, then the maximum possible coded copyabove run length is five.

In some examples, video decoder 30 may only determine the run mode(e.g., the palette mode in which the pixels are coded) for the pixelswithin dashed lines 280. Hence, in the worst case, video decoder 30makes the determination for BlockWidth-1 pixels. In some examples, videodecoder 30 may be configured to implement certain restrictions regardingthe maximum of number of pixels for which the run mode is checked. Forexample, video decoder 30 may only check the pixels within dashed lines280 if the pixels are in the same row as the current pixel. Videodecoder 30 may infer that all other pixels within dashed lines 280 arenot coded as escape samples. The example in FIG. 6 assumes a rasterscanning order. The techniques however, may be applied to other scanningorders, such as vertical, horizontal traverse, and vertical traverse.

FIG. 7 is a flowchart illustrating an example process for encoding ablock of video data based on one or more block-level syntax elementsthat indicate whether any samples of a block are encoded as escapesamples, consistent with techniques of this disclosure. The process ofFIG. 7 is generally described as being performed by video encoder 20 forpurposes of illustration, although a variety of other processors mayalso carry out the process shown in FIG. 7.

In the example of FIG. 7, video encoder 20 determines a palette forencoding a block of video data (300). In some examples, video encoder 20may determine the palette-based palettes of one or more previouslyencoded blocks, e.g., using a palette predictor. Video encoder 20 mayalso determine the size of the palette (302). For example, video encoder20 may determine a number of entries in the determined palette.

Video encoder 20 may determine whether the palette size is zero (304).Based on the palette size being equal to zero (the yes branch of step304), video encoder 20 may determine that block-level escape syntax thatindicates that at least one sample of the block is an escape sample(306). In one example, video encoder 20 may determine that a block-levelescape flag is equal to one. Video encoder 20 then encodes all samplesof the block as escape samples and without encoding other syntaxelements for the block (308). For example, video encoder 20 may notencode an indication of the block-level escape flag, because theblock-level escape syntax element may only be encoded in the bitstreamwhen the palette size is greater than zero. In addition, video encoder20 may not encode other data for palette indices of the block, such assyntax that indicates a palette mode (e.g., Value or CopyFromTop),syntax associated with determining runs, syntax associated with paletteindices, and any other related syntax.

If the palette size is not zero (the no branch of step 304), videoencoder 20 may determine whether any samples of the block are encoded asescape samples (310). Based on determining that at least one sample ofthe block is encoded as an escape sample (the yes branch of step 310)video encoder 20 may determine block-level escape syntax that indicatesthat at least one sample of the block is encoded as an escape sample andencode an indication of the block-level syntax in a bitstream with theblock (312). For example, video encoder 20 may set a block-level escapeflag equal to one and encode an indication of the escape flag in thebitstream. Video encoder 20 also encodes the block with palette codingmodes, including encoding at least one pixel of the block as an escapesample (314).

Based on no samples being encoded as escape samples (the no branch ofstep 310), video encoder 20 may determine whether a palette size of thepalette for the block is equal to one (316). Based on determining thatthe palette size is not equal to one (the no branch of step 316), videoencoder 20 may determine block-level escape syntax that indicates thatno samples of the block are encoded as escape samples encode anindication of the block-level escape syntax in the bitstream (318). Forexample, video encoder 20 may set a block-level escape flag equal tozero and encode an indication of the escape flag in the bitstream. Videoencoder 20 also encodes the block using palette coding modes but withoutencoding any escape samples (320). For example, video encoder 20 mayencode palette indices of the block using CopyFromTop or Value modes andencode syntax associated with the use of such modes, e.g., syntax thatindicates modes, palette indices, runs, and the like.

Based on determining that the palette size is equal to one (the yesbranch of step 316), video encoder 20 may determine block-level escapesyntax that indicates that no samples of the block are coded as escapesamples and encode an indication of the block-level escape syntax in thebitstream (322). For example, video encoder 20 may set a block-levelescape flag equal to zero and encode an indication of the escape flag inthe bitstream. Video encoder 20 also encodes an indication that allsamples of the block have the same index value and without encodingother syntax (324). For example, video encoder 20 may skip the encodingof syntax associated with the use of palette modes, e.g., syntax thatindicates modes, palette indices, runs, and the like.

FIG. 8 is a flowchart illustrating an example process for decoding ablock of video data based on one or more block-level syntax elementsthat indicate whether any samples of a block are decoded as escapesamples, consistent with techniques of this disclosure. The process ofFIG. 8 is generally described as being performed by video decoder 30 forpurposes of illustration, although a variety of other processors mayalso carry out the process shown in FIG. 8.

In the example of FIG. 8, video decoder 30 determines a palette fordecoding a block of video data (340). In some examples, video decoder 30may determine the palette-based palettes of one or more previouslyencoded blocks. e.g., using a palette predictor signaled in a bitstreambeing decoded. Video decoder 30 may also determine the size of thepalette (342). For example, video decoder 30 may determine a number ofentries in the determined palette.

Video decoder 30 may determine whether the palette size is zero (344).Based on the palette size being equal to zero (the yes branch of step344), video decoder 30 may determine block-level escape syntax thatindicates that at least one sample of the block is encoded as an escapesample (346). In one example, video decoder 30 may determine that ablock-level escape flag is equal to one. For example, video decoder 30may infer that the block-level escape flag is equal to one withoutdecoding the flag from the bitstream, because the block-level escapesyntax element may only be coded in the bitstream when the palette sizeis greater than zero. Video decoder 30 then decodes all samples of theblock using the color associated with the escape sample and withoutdecoding other syntax elements for the block (e.g., other than the colorvalue(s) associated with the escape sample) (348). For example, as notedabove, video decoder 30 may not decode an indication of the block-levelescape flag. In addition, video decoder 30 may not decode other data forpalette indices of the block, such as syntax that indicates a palettemode (e.g., Value or CopyFromTop), syntax indicating index values,syntax associated with determining runs, and any other related syntax.

If the palette size is not zero (the no branch of step 344), videodecoder 30 may decode block-level escape syntax from the bitstream anddetermine a value of the block-level syntax (350). For example, videodecoder 30 may decode a block-level escape flag and determine whetherthe value is equal to zero or one.

Video decoder 30 may then determine whether any samples of the block arecoded as an escape sample, e.g., based on the decoded syntax (352).Based on determining that at least one sample of the block is encoded asan escape sample (the yes branch of step 352), video decoder 30 maydecode the block with palette coding modes, including decoding at leastone sample of the block as an escape sample (354). Video decoder 30 mayalso decode at least one color value corresponding to the escapesamples.

Based on no samples are encoded as escape samples (the no branch of step352), video decoder 30 may determine whether a palette size of thepalette for the block is equal to one (356). Based on determining thatthe palette size is not equal to one (the no branch of step 356), videodecoder 30 may decode the block using palette coding modes but withoutdecoding any escape samples (358). For example, video decoder 30 maydecode palette indices of the block using CopyFromTop or Value modes anddecode syntax associated with the use of such modes, e.g., syntax thatindicates modes, palette indices, runs, and the like.

Based on determining that the palette size is equal to one (the yesbranch of step 356), video decoder 30 may decode the block using thepalette entry of the palette (e.g., the only entry present in thepalette) and without decoding other syntax (360). For example, videodecoder 30 may skip the decoding of syntax associated with the use ofpalette modes, e.g., syntax that indicates modes, palette indices, runs,and the like.

FIG. 9 is a flowchart illustrating an example process for encoding ablock of video data based on one or more syntax elements that indicate amaximum palette size and a maximum palette predictor size, consistentwith techniques of this disclosure. The process of FIG. 9 is generallydescribed as being performed by video encoder 20 for purposes ofillustration, although a variety of other processors may also carry outthe process shown in FIG. 9.

In the example of FIG. 9, video encoder 20 to may determine a maximumsize of a palette for encoding a current block of video data in palettemode (380). For example, video encoder 20 may be configured to determinea maximum palette size based on a characteristic of the video data beingencoded. In some examples, video encoder 20 may determine a maximumpalette size based on a bit-depth of the data (e.g., an input bit-depthor a profile bit-depth), a block size of the block, a profile or levelassociated with the video data, or the like.

Video encoder 20 to may also determine a maximum palette predictor sizeof a palette predictor for creating a palette of the current block(382). For example, video encoder 20 may be configured to determine amaximum palette predictor size based on a characteristic of the videodata being encoded. In some examples, video encoder 20 may determine amaximum palette predictor size based on a bit-depth of the data (e.g.,an input bit-depth or a profile bit-depth), a block size of the block, aprofile or level associated with the video data, or the like.

Video encoder 20 also encodes data that indicates the maximum palettesize and/or the maximum palette predictor size in a bitstream thatincludes the current block (386). In some examples, video encoder 20 mayencode data indicating the maximum palette size and/or maximum palettepredictor size relative to one or more other values. For example,according to aspects of this disclosure, video encoder 20 may beconfigured to code data indicating the maximum palette predictor size asa delta (e.g., difference) between the maximum palette predictor sizeand the maximum palette size.

According to aspects of this disclosure, video encoder 20 may encode oneor more syntax elements that indicate the maximum palette size and/orthe maximum palette predictor size in a SPS. In other examples, videoencoder 20 may encode such syntax in another parameter set (e.g., aPPS), in a slice header of a slice that includes the current block, orelsewhere in the bitstream.

Video encoder 20 also encodes the current block in accordance with thedata that indicates the maximum palette size and/or the maximum palettepredictor size (388). For example, video encoder 20 may determine apalette that is limited by the maximum palette size and/or a palettepredictor that is limited by the maximum palette predictor size.

FIG. 10 is a flowchart illustrating an example process for encoding ablock of video data based on one or more syntax elements that indicate amaximum palette size and a maximum palette predictor size, consistentwith techniques of this disclosure. The process of FIG. 10 is generallydescribed as being performed by video decoder 30 for purposes ofillustration, although a variety of other processors may also carry outthe process shown in FIG. 10.

In the example of FIG. 10, video decoder 30 decodes data that indicatesa maximum palette size and/or a maximum palette predictor size from abitstream that includes a current block being decoded in palette mode(400). In some examples, video encoder 20 may encode data indicating themaximum palette size and/or maximum palette predictor size relative toone or more other values. For example, according to aspects of thisdisclosure, video encoder 20 may be configured to code data indicatingthe maximum palette predictor size as a delta (e.g., difference) betweenthe maximum palette predictor size and the maximum palette size.

According to aspects of this disclosure, video decoder 30 may decode oneor more syntax elements that indicate the maximum palette size and/orthe maximum palette predictor size from an SPS. In other examples, videodecoder 30 may decode such syntax from another parameter set (e.g., aPPS), from a slice header of a slice that includes the current block, orelsewhere in the bitstream.

Video decoder 30 to may determine the maximum size of a palette fordecoding the current block based on the decoded data (402). Videodecoder 30 to may also determine a maximum palette predictor size of apalette predictor for creating a palette for the current block based onthe data (404). Video decoder 30 also decodes the current block inaccordance with the data that indicates the maximum palette size and/orthe maximum palette predictor size (408). For example, video decoder 30may determine a palette that is limited by the maximum palette sizeand/or a palette predictor that is limited by the maximum palettepredictor size.

FIG. 11 is a flowchart illustrating an example process for coding(encoding or decoding) data that indicates a run length of a run ofpixels based a maximum potential run length, consistent with techniquesof this disclosure. The process of FIG. 11 is generally described asbeing performed by a video coder, such as video encoder 20 or videodecoder 30, for purposes of illustration, although a variety of otherprocessors may also carry out the process shown in FIG. 1I.

In the example of FIG. 11, the video coder may determine a palette modefor coding a current pixel (420). For example, the video coder maydetermine whether the current pixel is coded using a CopyFromTop mode, aValue mode, or another palette-based coding mode. The video coder alsodetermines a run length of a run for the current pixel (422). Forexample, the video coder determines the number of palette indices beingcoded with the palette index of the current pixel.

The video coder also determines a maximum run length for the run (424).For example, the video coder may determine a maximum run length for amaximum run of palette indices able to be coded with the palette indexof the current pixel. In an example, the video coder may determine anumber of pixels in the block of video data that includes the pixel. Thevideo coder may also determine a position of the current pixel in theblock based on a scanning order used to scan the palette indices. Thevideo coder then determines the maximum run length as the number ofpixels in the block minus the pixel position of the current pixel minusone.

The video coder also codes data that indicates the run length of the runbased on the determined maximum run length (426). For example, accordingto aspects of this disclosure, the video coder may code data thatindicates the run length using a TEGk code.

It is to be recognized that depending on the example, certain acts orevents of any of the techniques described herein can be performed in adifferent sequence, may be added, merged, or left out altogether (e.g.,not all described acts or events are necessary for the practice of thetechniques). Moreover, in certain examples, acts or events may beperformed concurrently, e.g., through multi-threaded processing,interrupt processing, or multiple processors, rather than sequentially.In addition, while certain aspects of this disclosure are described asbeing performed by a single module or unit for purposes of clarity, itshould be understood that the techniques of this disclosure may beperformed by a combination of units or modules associated with a videocoder.

Certain aspects of this disclosure have been described with respect tothe developing HEVC standard for purposes of illustration. However, thetechniques described in this disclosure may be useful for other videocoding processes, including other standard or proprietary video codingprocesses not yet developed.

The techniques described above may be performed by video encoder 20(FIGS. 1 and 2) and/or video decoder 30 (FIGS. 1 and 3), both of whichmay be generally referred to as a video coder. Likewise, video codingmay refer to video encoding or video decoding, as applicable.

While particular combinations of various aspects of the techniques aredescribed above, these combinations are provided merely to illustrateexamples of the techniques described in this disclosure. Accordingly,the techniques of this disclosure should not be limited to these examplecombinations and may encompass any conceivable combination of thevarious aspects of the techniques described in this disclosure.

In one or more examples, the functions described may be implemented inhardware, software, firmware, or any combination thereof. If implementedin software, the functions may be stored on or transmitted over, as oneor more instructions or code, a computer-readable medium and executed bya hardware-based processing unit.

Computer-readable media may include computer-readable storage media,which corresponds to a tangible medium such as data storage media, orcommunication media including any medium that facilitates transfer of acomputer program from one place to another, e.g., according to acommunication protocol. In this manner, computer-readable mediagenerally may correspond to (1) tangible computer-readable storage mediawhich is non-transitory or (2) a communication medium such as a signalor carrier wave. Data storage media may be any available media that canbe accessed by one or more computers or one or more processors toretrieve instructions, code and/or data structures for implementation ofthe techniques described in this disclosure. A computer program productmay include a computer-readable medium.

By way of example, and not limitation, such computer-readable storagemedia can comprise RAM, ROM, EEPROM, CD-ROM or other optical diskstorage, magnetic disk storage, or other magnetic storage devices, flashmemory, or any other medium that can be used to store desired programcode in the form of instructions or data structures and that can beaccessed by a computer. Also, any connection is properly termed acomputer-readable medium. For example, if instructions are transmittedfrom a website, server, or other remote source using a coaxial cable,fiber optic cable, twisted pair, digital subscriber line (DSL), orwireless technologies such as infrared, radio, and microwave, then thecoaxial cable, fiber optic cable, twisted pair, DSL, or wirelesstechnologies such as infrared, radio, and microwave are included in thedefinition of medium. It should be understood, however, thatcomputer-readable storage media and data storage media do not includeconnections, carrier waves, signals, or other transient media, but areinstead directed to non-transient, tangible storage media. Disk anddisc, as used herein, includes compact disc (CD), laser disc, opticaldisc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wheredisks usually reproduce data magnetically, while discs reproduce dataoptically with lasers. Combinations of the above should also be includedwithin the scope of computer-readable media.

Instructions may be executed by one or more processors, such as one ormore digital signal processors (DSPs), general purpose microprocessors,application specific integrated circuits (ASICs), field programmablelogic arrays (FPGAs), or other equivalent integrated or discrete logiccircuitry. Accordingly, the term “processor,” as used herein may referto any of the foregoing structure or any other structure suitable forimplementation of the techniques described herein. In addition, in someaspects, the functionality described herein may be provided withindedicated hardware and/or software modules configured for encoding anddecoding, or incorporated in a combined codec. Also, the techniquescould be fully implemented in one or more circuits or logic elements.

The techniques of this disclosure may be implemented in a wide varietyof devices or apparatuses, including a wireless handset, an integratedcircuit (IC) or a set of ICs (e.g., a chip set). Various components,modules, or units are described in this disclosure to emphasizefunctional aspects of devices configured to perform the disclosedtechniques, but do not necessarily require realization by differenthardware units. Rather, as described above, various units may becombined in a codec hardware unit or provided by a collection ofinteroperative hardware units, including one or more processors asdescribed above, in conjunction with suitable software and/or firmware.

Various examples have been described. These and other examples arewithin the scope of the following claims.

What is claimed is:
 1. A method of processing video data, the methodcomprising: determining a value of a block-level syntax element thatindicates, for all samples of a block of video data, whether at leastone respective sample of the block is coded based on a color value ofthe at least one respective sample not being included in a palette ofcolors for coding the block of video data; and coding the block of videodata based on the value.
 2. The method of claim 1, wherein determiningthe value of the block-level syntax element comprises determining avalue ofa block-level escape flag that indicates at least one respectivesample is coded as an escape sample.
 3. The method of claim 2, whereinthe block of video data comprises a coding unit (CU) of video data, andwherein determining the value of the block-level escape flag comprisesdetermining the value of the block-level escape flag for the CU.
 4. Themethod of claim 3, further comprising: coding palette entries of thepalette for the CU; and coding the block-level escape flag for the CUafter coding the palette entries of the palette.
 5. The method of claim3, further comprising: coding the block-level escape flag for the CU;and coding palette entries of the palette for the CU after coding theblock-level escape flag for the CU.
 6. The method of claim 3, furthercomprising conditionally coding the block-level escape flag for the CUbased on a size of the CU, wherein conditionally coding the block-levelescape flag for the CU comprises only coding the block-level escape flagwhen the size of the CU exceeds a threshold size.
 7. The method of claim1, wherein the determined value of the block-level syntax elementindicates that all samples of the block are coded with at least onecolor of the palette, and wherein the method further comprises:determining a palette size that indicates a number of palette indices ofthe palette for the block; and wherein, based on the determined palettesize being one index, coding the block comprises coding all samples ofthe block based on the one index and without coding any other syntax forthe block that indicates palette indices for the block.
 8. The method ofclaim 7, wherein coding the block without coding any other syntax forthe block that indicates palette indices for the block comprises codingthe block without coding at least one of data that indicates a palettemode of samples of the block, data that indicates index values of thepalette for the block, and data that indicates a run of palette indicesof the palette.
 9. The method of claim 1, further comprising:determining a palette size that indicates a number of palette indices ofthe palette for the block; and wherein, based on the determined palettesize being zero indices, determining the value of the block-level syntaxelement comprises inferring the value of the syntax element, whereininferring the value of the block-level syntax element comprisesdetermining the value of the block-level syntax element without codingthe of the block-level syntax element, and wherein the inferred value ofthe syntax element indicates that all samples of the block are codedbased on the color that is not included in the palette.
 10. The methodof claim 9, wherein, based on the determined palette size being zeroindices, coding the block comprises coding all samples of the block withthe color value that is not included in the palette and without codingany other syntax for the block that indicates palette indices for theblock.
 11. The method of claim 10, wherein coding the block withoutcoding any other syntax for the block that indicates palette indices forthe block comprises coding the block without coding at least one of datathat indicates a palette mode of samples of the block, data thatindicates palette indices of the palette for the block, and data thatindicates a run of palette indices of the palette.
 12. The method ofclaim 1, wherein coding comprises encoding, and wherein encoding theblock of video data based on the value comprises: based on thedetermined value indicating that at least one respective sample of theblock is not encoded based on the color value of the at least onerespective sample not being included in the palette of colors,determining respective index values for the samples of the block,wherein the respective index values identify respective entries of thepalette; based on the determined value indicating that at least onerespective sample of the block is encoded based on the color value ofthe at least one respective sample not being included in the palette ofcolors, determining respective index values for the samples of theblock, wherein the respective index values identify respective entriesof the palette, and wherein one of the respective index values indicatesan escape sample; and encoding the respective index values in an encodedbitstream.
 13. The method of claim 1, wherein coding comprises decoding,and wherein decoding the block of video data based on the valuecomprises: obtaining, from an encoded bitstream, respective index valuesfor samples of the block, wherein the respective index values identifyan entry of the palette; based on the determined value indicating thatat least one respective sample of the block is not decoded based on thecolor value of the at least one respective sample not being included inthe palette of colors, determining values for the samples by matchingthe respective index values to at least one of the entries of thepalette; and based on the determined value indicating that at least onerespective sample of the block is decoded based on the color value ofthe at least one respective sample not being included in the palette ofcolors, determining values for the samples by matching the respectiveindex values to at least one of the entries of the palette and decodingat least one color value for the color value that is not included in thepalette of colors.
 14. A device for processing video data, the devicecomprising: a memory configured to store a block of samples of videodata; and one or more processors configured to: determine a value of ablock-level syntax element that indicates, for all the samples of theblock of video data, whether at least one respective sample of the blockis coded based on a color value of the at least one respective samplenot being included in a palette of colors for coding the block of videodata; and code the block of video data based on the value.
 15. Thedevice of claim 14, wherein to determine the value of the block-levelsyntax element, the one or more processors are configured to determine avalue of a block-level escape flag that indicates at least onerespective sample is coded as an escape sample.
 16. The device of claim15, wherein the block of video data comprises a coding unit (CU) ofvideo data, and wherein to determine the value of the block-level escapeflag, the one or more processors are configured to determine the valueof the block-level escape flag for the CU.
 17. The device of claim 16,wherein the one or more processors are further configured to: codepalette entries of the palette for the CU; and code the block-levelescape flag for the CU after coding the palette entries of the palette.18. The device of claim 16, wherein the one or more processors arefurther configured to: code the block-level escape flag for the CU; andcode palette entries of the palette for the CU after coding theblock-level escape flag for the CU.
 19. The device of claim 16, whereinthe one or more processors are further configured to conditionally codethe block-level escape flag for the CU based on a size of the CU,wherein to conditionally code the block-level escape flag for the CU,the one or more processors are configured to only code the block-levelescape flag when the size of the CU exceeds a threshold size.
 20. Thedevice of claim 14, wherein the determined value of the block-levelsyntax element indicates that all samples of the block are coded with atleast one color of the palette, and wherein the one or more processorsare further configured to: determine a palette size that indicates anumber of palette indices of the palette for the block; and wherein,based on the determined palette size being one index, to code the block,the one or more processors are configured to code all samples of theblock based on the one index and without coding any other syntax for theblock that indicates palette indices for the block.
 21. The device ofclaim 20, wherein to code the block without coding any other syntax forthe block that indicates palette indices for the block, the one or moreprocessors are configured to code the block without coding at least oneof data that indicates a palette mode of samples of the block, data thatindicates index values of the palette for the block, and data thatindicates a run of palette indices of the palette.
 22. The device ofclaim 14, wherein the one or more processors are further configured to:determine a palette size that indicates a number of palette indices ofthe palette for the block; and wherein, based on the determined palettesize being zero indices, to determine the value of the block-levelsyntax element, the one or more processors are configured to infer thevalue of the syntax element including determining the value of theblock-level syntax element without coding the of the block-level syntaxelement, and wherein the inferred value of the syntax element indicatesthat all samples of the block are coded based on the color that is notincluded in the palette.
 23. The device of claim 22, wherein, based onthe determined palette size being zero indices, to code the block, theone or more processors are configured to code all samples of the blockwith the color value that is not included in the palette and withoutcoding any other syntax for the block that indicates palette indices forthe block.
 24. The device of claim 23, wherein to code the block withoutcoding any other syntax for the block that indicates palette indices forthe block, the one or more processors are configured to code the blockwithout coding at least one of data that indicates a palette mode ofsamples of the block, data that indicates palette indices of the palettefor the block, and data that indicates a run of palette indices of thepalette.
 25. The device of claim 14, wherein to code, the one or moreprocessors are configured to encode, and wherein to encode the block ofvideo data based on the value, the one or more processors are configuredto: based on the determined value indicating that at least onerespective sample of the block is not encoded based on the color valueof the at least one respective sample not being included in the paletteof colors, determine respective index values for the samples of theblock, wherein the respective index values identify respective entriesof the palette; based on the determined value indicating that at leastone respective sample of the block is encoded based on the color valueof the at least one respective sample not being included in the paletteof colors, determine respective index values for the samples of theblock, wherein the respective index values identify respective entriesof the palette, and wherein one of the respective index values indicatesan escape sample; and encode the respective index values in an encodedbitstream.
 26. The device of claim 14, wherein to code, the one or moreprocessors are configured to decode, and wherein to decode the block ofvideo data based on the value, the one or more processors are configuredto: obtain, from an encoded bitstream, respective index values forsamples of the block, wherein the respective index values identify anentry of the palette; based on the determined value indicating that atleast one respective sample of the block is not decoded based on thecolor value of the at least one respective sample not being included inthe palette of colors, determine values for the samples by matching therespective index values to at least one of the entries of the palette;and based on the determined value indicating that at least onerespective sample of the block is decoded based on the color value ofthe at least one respective sample not being included in the palette ofcolors, determine values for the samples by matching the respectiveindex values to at least one of the entries of the palette and decodingat least one color value for the color value that is not included in thepalette of colors.
 27. The device of claim 26, further comprising adisplay configured to display the decoded block.
 28. The device of claim14, wherein the device comprises at least one of: an integrated circuit;a microprocessor; or a wireless communication device.
 29. An apparatusfor processing video data, the apparatus comprising: means fordetermining a value of a block-level syntax element that indicates, forall samples of a block of video data, whether at least one respectivesample of the block is coded based on a color value of the at least onerespective sample not being included in a palette of colors for codingthe block of video data; and means for coding the block of video databased on the value.
 30. A non-transitory computer-readable medium havingstored thereon instructions that, when executed, cause one or moreprocessors to: determine a value of a block-level syntax element thatindicates, for all samples of a block of video data, whether at leastone respective sample of the block is coded based on a color value ofthe at least one respective sample not being included in a palette ofcolors for coding the block of video data; and code the block of videodata based on the value.